[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:20:43 dist-test-slave-k84h krb5kdc[1329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:20:43 dist-test-slave-k84h krb5kdc[1329](info): set up 2 sockets
Jun 19 22:20:43 dist-test-slave-k84h krb5kdc[1329](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 22:20:44.267958 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:20:44 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:20:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:20:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:20:44.390229 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:38517
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:38517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:44.513836 1342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:44.514183 1342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:44.514266 1342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:44.518464 1342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:20:44.518574 1342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:44.518611 1342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:44.518666 1342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:20:44.518702 1342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:20:44.524663 1342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:38517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:38517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:44.525873 1342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:44.526961 1342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:44.536046 1347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:44.536005 1348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:44.536019 1342 server_base.cc:1034] running on GCE node
W20260619 22:20:44.536064 1350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:44.536933 1342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:44.538089 1342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:44.539249 1342 hybrid_clock.cc:648] HybridClock initialized: now 1781907644539234 us; error 30 us; skew 500 ppm
Jun 19 22:20:44 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:44.542097 1342 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:20:44.543115 1342 webserver.cc:460] Webserver started at http://127.1.74.190:44045/ using document root <none> and password file <none>
I20260619 22:20:44.543644 1342 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:44.543707 1342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:44.543877 1342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:44.545993 1342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "bf92553de00243efab9bf00bcdab2a2f"
format_stamp: "Formatted at 2026-06-19 22:20:44 on dist-test-slave-k84h"
server_key: "7ca52e6c8840d57c84efb3a4b7c166da"
server_key_iv: "6f216df87a4ff72903c265d6be3fab33"
server_key_version: "encryptionkey@0"
I20260619 22:20:44.546561 1342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "bf92553de00243efab9bf00bcdab2a2f"
format_stamp: "Formatted at 2026-06-19 22:20:44 on dist-test-slave-k84h"
server_key: "7ca52e6c8840d57c84efb3a4b7c166da"
server_key_iv: "6f216df87a4ff72903c265d6be3fab33"
server_key_version: "encryptionkey@0"
I20260619 22:20:44.550611 1342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:20:44.553319 1357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:44.554661 1342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 22:20:44.554811 1342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "bf92553de00243efab9bf00bcdab2a2f"
format_stamp: "Formatted at 2026-06-19 22:20:44 on dist-test-slave-k84h"
server_key: "7ca52e6c8840d57c84efb3a4b7c166da"
server_key_iv: "6f216df87a4ff72903c265d6be3fab33"
server_key_version: "encryptionkey@0"
I20260619 22:20:44.554996 1342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:44.597366 1342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:44.600955 1342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:44.601193 1342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:44.608666 1342 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:38517
I20260619 22:20:44.608717 1409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:38517 every 8 connection(s)
I20260619 22:20:44.609683 1342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:20:44.612635 1410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:44.617889 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 1342
I20260619 22:20:44.618072 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:20:44.618875 1410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f: Bootstrap starting.
I20260619 22:20:44.619753 1322 external_mini_cluster.cc:1428] Setting key 568f0446a26aff56aec5998e9deb4cf0
I20260619 22:20:44.622120 1410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:44.623111 1410 log.cc:824] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:44.625223 1410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f: No bootstrap required, opened a new log
I20260619 22:20:44.629191 1410 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92553de00243efab9bf00bcdab2a2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 38517 } }
I20260619 22:20:44.629474 1410 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:44.629559 1410 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf92553de00243efab9bf00bcdab2a2f, State: Initialized, Role: FOLLOWER
I20260619 22:20:44.630060 1410 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92553de00243efab9bf00bcdab2a2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 38517 } }
I20260619 22:20:44.630228 1410 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:20:44.630321 1410 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:20:44.630451 1410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:44.631562 1410 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92553de00243efab9bf00bcdab2a2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 38517 } }
I20260619 22:20:44.631923 1410 leader_election.cc:302] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf92553de00243efab9bf00bcdab2a2f; no voters:
I20260619 22:20:44.632261 1410 leader_election.cc:288] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:20:44.632400 1415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:44.632609 1415 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [term 1 LEADER]: Becoming Leader. State: Replica: bf92553de00243efab9bf00bcdab2a2f, State: Running, Role: LEADER
I20260619 22:20:44.632921 1415 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92553de00243efab9bf00bcdab2a2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 38517 } }
I20260619 22:20:44.633572 1410 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:20:44.634158 1417 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf92553de00243efab9bf00bcdab2a2f. Latest consensus state: current_term: 1 leader_uuid: "bf92553de00243efab9bf00bcdab2a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92553de00243efab9bf00bcdab2a2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 38517 } } }
I20260619 22:20:44.634301 1417 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:44.634593 1416 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf92553de00243efab9bf00bcdab2a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf92553de00243efab9bf00bcdab2a2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 38517 } } }
I20260619 22:20:44.634668 1416 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:44.634979 1420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:20:44.638058 1420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 22:20:44 dist-test-slave-k84h krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:44.643872 1420 catalog_manager.cc:1347] Generated new cluster ID: efb35ec0452b469eb0eda6a7a414015c
I20260619 22:20:44.643939 1420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:20:44.648216 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:44.623367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.1.74.190:38517)
0619 22:20:44.623836 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:44.623859 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:44.625135 (+ 1276us) server_negotiation.cc:407] Connection header received
0619 22:20:44.628802 (+ 3667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:44.628821 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:44.629171 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:44.629543 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:44.632107 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:44.633655 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:44.635562 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:44.636277 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:44.636435 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:44.641797 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:44.641804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:44.641819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:44.641853 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:44.644136 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:44.644858 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:44.644863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:44.644869 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:44.645034 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:44.645461 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:44.645466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:44.645470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:44.646017 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:44.646240 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:44.646897 (+ 657us) server_negotiation.cc:299] Negotiation successful
0619 22:20:44.647208 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
I20260619 22:20:44.678655 1420 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:20:44.679746 1420 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:20:44.685119 1420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf92553de00243efab9bf00bcdab2a2f: Generated new TSK 0
I20260619 22:20:44.685845 1420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:20:44.717974 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:38517
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:44.836447 1436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:44.836758 1436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:44.836805 1436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:44.841024 1436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:44.841099 1436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:44.841163 1436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:20:44.846804 1436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:38517
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:44.847970 1436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:44.849032 1436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:44.859429 1442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:44.859409 1441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:44.859409 1444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:44.859849 1436 server_base.cc:1034] running on GCE node
I20260619 22:20:44.860376 1436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:44.861090 1436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:44.862286 1436 hybrid_clock.cc:648] HybridClock initialized: now 1781907644862267 us; error 31 us; skew 500 ppm
Jun 19 22:20:44 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:44.865435 1436 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:20:44.866364 1436 webserver.cc:460] Webserver started at http://127.1.74.129:42331/ using document root <none> and password file <none>
I20260619 22:20:44.867007 1436 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:44.867075 1436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:44.867259 1436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:44.869565 1436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "a0391bea4e634322b5cb0d621b8903a7"
format_stamp: "Formatted at 2026-06-19 22:20:44 on dist-test-slave-k84h"
server_key: "256d47c218d8a541e40faf7564ad9b22"
server_key_iv: "3e826cfe40b03f6969fba76394721dc2"
server_key_version: "encryptionkey@0"
I20260619 22:20:44.870159 1436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "a0391bea4e634322b5cb0d621b8903a7"
format_stamp: "Formatted at 2026-06-19 22:20:44 on dist-test-slave-k84h"
server_key: "256d47c218d8a541e40faf7564ad9b22"
server_key_iv: "3e826cfe40b03f6969fba76394721dc2"
server_key_version: "encryptionkey@0"
I20260619 22:20:44.874382 1436 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:20:44.877323 1451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:44.879277 1436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:20:44.879458 1436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "a0391bea4e634322b5cb0d621b8903a7"
format_stamp: "Formatted at 2026-06-19 22:20:44 on dist-test-slave-k84h"
server_key: "256d47c218d8a541e40faf7564ad9b22"
server_key_iv: "3e826cfe40b03f6969fba76394721dc2"
server_key_version: "encryptionkey@0"
I20260619 22:20:44.879607 1436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:44.902665 1436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:44.906093 1436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:44.906333 1436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:44.907027 1436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:20:44.907888 1436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:44.907987 1436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:44.908080 1436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:44.908124 1436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:44.917497 1436 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:43297
I20260619 22:20:44.917520 1564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:43297 every 8 connection(s)
I20260619 22:20:44.918409 1436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:20:44.925304 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 1436
I20260619 22:20:44.925393 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:20:44.925647 1322 external_mini_cluster.cc:1428] Setting key 0f476de832f28f6bce25855f4e87b108
Jun 19 22:20:44 dist-test-slave-k84h krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907644, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:44.933269 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:44.920360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:57715 (local address 127.1.74.190:38517)
0619 22:20:44.920503 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:44.920507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:44.921330 (+ 823us) server_negotiation.cc:407] Connection header received
0619 22:20:44.922724 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:44.922727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:44.922790 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:44.922937 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:44.924134 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:44.924691 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:44.926009 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:44.926305 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:44.926413 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:44.929403 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:44.929410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:44.929414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:44.929448 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:44.931110 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:44.931675 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:44.931679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:44.931681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:44.931764 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:44.932122 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:44.932125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:44.932127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:44.932317 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:44.932441 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:44.932976 (+ 535us) server_negotiation.cc:299] Negotiation successful
0619 22:20:44.933105 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:20:44.934278 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:44.920749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:38517 (local address 127.1.74.129:57715)
0619 22:20:44.921168 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:44.921211 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:44.922489 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:44.923111 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:44.923118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:44.923479 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:44.923963 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:44.923976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:44.924832 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:44.924841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:44.925882 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:44.925892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:44.926483 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:44.926487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:44.926703 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:44.927438 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:44.927454 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:44.929201 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:44.931248 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:44.931259 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:44.931276 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:44.931566 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:44.931857 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:44.931861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:44.931863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:44.932019 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:44.932441 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:44.932449 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:44.932716 (+ 267us) client_negotiation.cc:769] Sending connection context
0619 22:20:44.932936 (+ 220us) client_negotiation.cc:240] Negotiation successful
0619 22:20:44.933283 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:20:44.935624 1565 heartbeater.cc:347] Connected to a master server at 127.1.74.190:38517
I20260619 22:20:44.935892 1565 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:44.936450 1565 heartbeater.cc:511] Master 127.1.74.190:38517 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:20:44.938126 1374 ts_manager.cc:194] Registered new tserver with Master: a0391bea4e634322b5cb0d621b8903a7 (127.1.74.129:43297)
I20260619 22:20:44.939486 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:57715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:20:44.953923 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:38517
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:45.068483 1570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:45.068754 1570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:45.068799 1570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:45.072901 1570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:45.072966 1570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:45.073037 1570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:20:45.078292 1570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:38517
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:45.079346 1570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:45.080346 1570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:45.090283 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:45.090283 1578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:45.090322 1576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:45.090662 1570 server_base.cc:1034] running on GCE node
I20260619 22:20:45.091207 1570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:45.091971 1570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:45.093173 1570 hybrid_clock.cc:648] HybridClock initialized: now 1781907645093159 us; error 28 us; skew 500 ppm
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:45.096029 1570 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:20:45.096988 1570 webserver.cc:460] Webserver started at http://127.1.74.130:34431/ using document root <none> and password file <none>
I20260619 22:20:45.097523 1570 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:45.097586 1570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:45.097752 1570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:45.099887 1570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "55606b19c82347f7879b4b38410a9a9f"
format_stamp: "Formatted at 2026-06-19 22:20:45 on dist-test-slave-k84h"
server_key: "50ce293b2010628e844f32684bc0a201"
server_key_iv: "60816be0d762a46b38cadf83f21f72f9"
server_key_version: "encryptionkey@0"
I20260619 22:20:45.100433 1570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "55606b19c82347f7879b4b38410a9a9f"
format_stamp: "Formatted at 2026-06-19 22:20:45 on dist-test-slave-k84h"
server_key: "50ce293b2010628e844f32684bc0a201"
server_key_iv: "60816be0d762a46b38cadf83f21f72f9"
server_key_version: "encryptionkey@0"
I20260619 22:20:45.104349 1570 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:20:45.106822 1585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:45.108148 1570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:20:45.108250 1570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "55606b19c82347f7879b4b38410a9a9f"
format_stamp: "Formatted at 2026-06-19 22:20:45 on dist-test-slave-k84h"
server_key: "50ce293b2010628e844f32684bc0a201"
server_key_iv: "60816be0d762a46b38cadf83f21f72f9"
server_key_version: "encryptionkey@0"
I20260619 22:20:45.108342 1570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:45.119105 1570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:45.121917 1570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:45.122058 1570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:45.122563 1570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:20:45.123358 1570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:45.123404 1570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:45.123442 1570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:45.123458 1570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:45.132583 1570 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:36051
I20260619 22:20:45.132598 1698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:36051 every 8 connection(s)
I20260619 22:20:45.133554 1570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:20:45.140211 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 1570
I20260619 22:20:45.140311 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:20:45.140569 1322 external_mini_cluster.cc:1428] Setting key 7ae403110a3a48a4ae65184261ea882b
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:45.148993 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:45.135627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:57657 (local address 127.1.74.190:38517)
0619 22:20:45.135770 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:45.135774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:45.136611 (+ 837us) server_negotiation.cc:407] Connection header received
0619 22:20:45.138000 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:45.138004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:45.138062 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:45.138187 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:45.139927 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.140554 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.141780 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.142000 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.142122 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:45.144708 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:45.144715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:45.144721 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:45.144754 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:45.146482 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.147068 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.147073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.147078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.147190 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.147547 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.147551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.147556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.147838 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:45.147956 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:45.148637 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 22:20:45.148779 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 22:20:45.149780 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:45.136032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:38517 (local address 127.1.74.130:57657)
0619 22:20:45.136459 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:45.136501 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:45.137778 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:45.138373 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:45.138380 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:45.138783 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:45.139774 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:45.139790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.140723 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.140726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:45.141646 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:45.141656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.142155 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.142159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:45.142338 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:45.143000 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:45.143017 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:45.144534 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:45.146610 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:45.146615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:45.146628 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:45.146967 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:45.147292 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:45.147294 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:45.147295 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:45.147438 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:45.147962 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:45.147969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:45.148310 (+ 341us) client_negotiation.cc:769] Sending connection context
0619 22:20:45.148522 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 22:20:45.148815 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:20:45.151149 1699 heartbeater.cc:347] Connected to a master server at 127.1.74.190:38517
I20260619 22:20:45.151414 1699 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:45.151942 1699 heartbeater.cc:511] Master 127.1.74.190:38517 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:20:45.153083 1374 ts_manager.cc:194] Registered new tserver with Master: 55606b19c82347f7879b4b38410a9a9f (127.1.74.130:36051)
I20260619 22:20:45.153744 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:57657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:20:45.169811 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:38517
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:45.287714 1704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:45.288002 1704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:45.288048 1704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:45.292066 1704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:45.292127 1704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:45.292188 1704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:20:45.297487 1704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:38517
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:45.298518 1704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:45.299494 1704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:45.308979 1710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:45.309033 1709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:45.308979 1712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:45.309280 1704 server_base.cc:1034] running on GCE node
I20260619 22:20:45.309689 1704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:45.310350 1704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:45.311532 1704 hybrid_clock.cc:648] HybridClock initialized: now 1781907645311514 us; error 30 us; skew 500 ppm
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:45.314325 1704 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:20:45.315340 1704 webserver.cc:460] Webserver started at http://127.1.74.131:42483/ using document root <none> and password file <none>
I20260619 22:20:45.315914 1704 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:45.316015 1704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:45.316250 1704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:45.318316 1704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "ac222857627c4c07a3d466c2b1363d17"
format_stamp: "Formatted at 2026-06-19 22:20:45 on dist-test-slave-k84h"
server_key: "8eef7aac1fed839789a61e15274e99b2"
server_key_iv: "a9c75f91a9e452431f762e3faaaea141"
server_key_version: "encryptionkey@0"
I20260619 22:20:45.318974 1704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "ac222857627c4c07a3d466c2b1363d17"
format_stamp: "Formatted at 2026-06-19 22:20:45 on dist-test-slave-k84h"
server_key: "8eef7aac1fed839789a61e15274e99b2"
server_key_iv: "a9c75f91a9e452431f762e3faaaea141"
server_key_version: "encryptionkey@0"
I20260619 22:20:45.323143 1704 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:20:45.325762 1719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:45.327196 1704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:20:45.327333 1704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "ac222857627c4c07a3d466c2b1363d17"
format_stamp: "Formatted at 2026-06-19 22:20:45 on dist-test-slave-k84h"
server_key: "8eef7aac1fed839789a61e15274e99b2"
server_key_iv: "a9c75f91a9e452431f762e3faaaea141"
server_key_version: "encryptionkey@0"
I20260619 22:20:45.327455 1704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:45.357096 1704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:45.360371 1704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:45.360582 1704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:45.361209 1704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:20:45.362059 1704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:45.362136 1704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:45.362203 1704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:45.362247 1704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:45.371826 1704 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:36035
I20260619 22:20:45.371843 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:36035 every 8 connection(s)
I20260619 22:20:45.372728 1704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:20:45.377413 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 1704
I20260619 22:20:45.377528 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:20:45.377820 1322 external_mini_cluster.cc:1428] Setting key a4c5508635c7a9bda38c343f0d64b398
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:45.387316 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:45.374911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:51811 (local address 127.1.74.190:38517)
0619 22:20:45.375054 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:45.375060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:45.375932 (+ 872us) server_negotiation.cc:407] Connection header received
0619 22:20:45.377267 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:45.377270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:45.377330 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:45.377430 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:45.378955 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.379641 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.380919 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.381141 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.381251 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:45.383741 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:45.383745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:45.383750 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:45.383780 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:45.385267 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.385778 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.385781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.385782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.385868 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.386191 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.386194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.386196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.386383 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:45.386471 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:45.387049 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 22:20:45.387177 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 22:20:45.388239 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:45.375345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:38517 (local address 127.1.74.131:51811)
0619 22:20:45.375786 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:45.375827 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:45.377030 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:45.377668 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:45.377679 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:45.378095 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:45.378687 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:45.378703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.379764 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.379767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:45.380797 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:45.380806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.381286 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.381290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:45.381458 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:45.382010 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:45.382025 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:45.383550 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:45.385379 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:45.385385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:45.385399 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:45.385684 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:45.385956 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:45.385958 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:45.385959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:45.386105 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:45.386473 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:45.386478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:45.386745 (+ 267us) client_negotiation.cc:769] Sending connection context
0619 22:20:45.386981 (+ 236us) client_negotiation.cc:240] Negotiation successful
0619 22:20:45.387268 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":99,"threads_started":1}
I20260619 22:20:45.389576 1833 heartbeater.cc:347] Connected to a master server at 127.1.74.190:38517
I20260619 22:20:45.389832 1833 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:45.390348 1833 heartbeater.cc:511] Master 127.1.74.190:38517 requested a full tablet report, sending...
I20260619 22:20:45.391666 1373 ts_manager.cc:194] Registered new tserver with Master: ac222857627c4c07a3d466c2b1363d17 (127.1.74.131:36035)
I20260619 22:20:45.392351 1373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:51811
I20260619 22:20:45.392868 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:20:45.420302 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:45.409544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48642 (local address 127.1.74.129:43297)
0619 22:20:45.409925 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:45.409943 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:45.410015 (+ 72us) server_negotiation.cc:407] Connection header received
0619 22:20:45.410069 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:45.410074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:45.410274 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:45.410391 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:45.411137 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.412073 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.413114 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.413525 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.413615 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:45.416390 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:45.416401 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:45.416416 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:45.416446 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:45.418145 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.418699 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.418705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.418707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.418794 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.419143 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.419146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.419148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.419452 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:45.419679 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:45.419884 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 22:20:45.420081 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":101,"threads_started":1}
W20260619 22:20:45.421367 1479 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48642
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:20:45.449290 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:45.440267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48652 (local address 127.1.74.129:43297)
0619 22:20:45.440448 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:45.440452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:45.440468 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:20:45.440620 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:45.440622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:45.440666 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:45.440752 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:45.441466 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.442044 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.442923 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:45.443164 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:45.443247 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:45.445785 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:45.445789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:45.445791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:45.445819 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:45.447426 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.447829 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.447832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.447833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.447906 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:45.448265 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:45.448268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:45.448269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:45.448491 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:45.448605 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:45.448933 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 22:20:45.449050 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 22:20:45.450376 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 1436
I20260619 22:20:45.456131 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 1570
I20260619 22:20:45.463819 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 1704
I20260619 22:20:45.469710 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 1342
2026-06-19T22:20:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2250 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1847](info): set up 2 sockets
Jun 19 22:20:45 dist-test-slave-k84h krb5kdc[1847](info): commencing operation
krb5kdc: starting...
W20260619 22:20:46.542531 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:20:46 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:20:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:20:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:20:46.670996 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:35323
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:35323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:46.783041 1860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:46.783385 1860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:46.783463 1860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:46.787860 1860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:20:46.787959 1860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:46.787999 1860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:46.788061 1860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:20:46.788102 1860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:20:46.793877 1860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:35323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:35323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:46.795151 1860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:46.796226 1860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:46.804839 1868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:46.804839 1865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:46.804862 1866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:46.804940 1860 server_base.cc:1034] running on GCE node
I20260619 22:20:46.805747 1860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:46.806835 1860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:46.808059 1860 hybrid_clock.cc:648] HybridClock initialized: now 1781907646808032 us; error 43 us; skew 500 ppm
Jun 19 22:20:46 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907646, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:46.810997 1860 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:20:46.812230 1860 webserver.cc:460] Webserver started at http://127.1.74.190:32973/ using document root <none> and password file <none>
I20260619 22:20:46.812947 1860 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:46.813037 1860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:46.813252 1860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:46.815444 1860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "bfa63cceb40a44748aa6ed96e5e31ea7"
format_stamp: "Formatted at 2026-06-19 22:20:46 on dist-test-slave-k84h"
server_key: "38213488a2e0384a1f2bdaf3f0b6230e"
server_key_iv: "d7a936bea8488b17728083a9d67cd2c5"
server_key_version: "encryptionkey@0"
I20260619 22:20:46.816045 1860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "bfa63cceb40a44748aa6ed96e5e31ea7"
format_stamp: "Formatted at 2026-06-19 22:20:46 on dist-test-slave-k84h"
server_key: "38213488a2e0384a1f2bdaf3f0b6230e"
server_key_iv: "d7a936bea8488b17728083a9d67cd2c5"
server_key_version: "encryptionkey@0"
I20260619 22:20:46.820115 1860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 22:20:46.822695 1875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:46.824038 1860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:20:46.824203 1860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "bfa63cceb40a44748aa6ed96e5e31ea7"
format_stamp: "Formatted at 2026-06-19 22:20:46 on dist-test-slave-k84h"
server_key: "38213488a2e0384a1f2bdaf3f0b6230e"
server_key_iv: "d7a936bea8488b17728083a9d67cd2c5"
server_key_version: "encryptionkey@0"
I20260619 22:20:46.824331 1860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:46.844054 1860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:46.847656 1860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:46.847877 1860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:46.855396 1860 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:35323
I20260619 22:20:46.855419 1927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:35323 every 8 connection(s)
I20260619 22:20:46.856398 1860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:20:46.857241 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 1860
I20260619 22:20:46.857333 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:20:46.857620 1322 external_mini_cluster.cc:1428] Setting key 120b1ea288ca12603501f0d9da9c0924
I20260619 22:20:46.859623 1928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:46.867070 1928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7: Bootstrap starting.
Jun 19 22:20:46 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:46.870190 1928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:46.871148 1928 log.cc:824] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:46.873525 1928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7: No bootstrap required, opened a new log
I20260619 22:20:46.873613 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:46.859099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.1.74.190:35323)
0619 22:20:46.859556 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:46.859574 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:46.859619 (+ 45us) server_negotiation.cc:407] Connection header received
0619 22:20:46.860460 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:46.860483 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:46.860898 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:46.861283 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:46.861953 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:46.863543 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:46.864495 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:46.865233 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:46.865412 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:46.868350 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:46.868358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:46.868378 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:46.868423 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:46.870448 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:46.870943 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:46.870949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:46.870955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:46.871097 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:46.871399 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:46.871402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:46.871403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:46.871876 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:46.872087 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:46.872374 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 22:20:46.872672 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":130,"threads_started":1}
I20260619 22:20:46.877367 1928 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 35323 } }
I20260619 22:20:46.877606 1928 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:46.877642 1928 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa63cceb40a44748aa6ed96e5e31ea7, State: Initialized, Role: FOLLOWER
I20260619 22:20:46.878069 1928 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 35323 } }
I20260619 22:20:46.878198 1928 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:20:46.878285 1928 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:20:46.878391 1928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:46.879477 1928 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 35323 } }
I20260619 22:20:46.879839 1928 leader_election.cc:302] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfa63cceb40a44748aa6ed96e5e31ea7; no voters:
I20260619 22:20:46.880194 1928 leader_election.cc:288] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:20:46.880317 1933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:46.880555 1933 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [term 1 LEADER]: Becoming Leader. State: Replica: bfa63cceb40a44748aa6ed96e5e31ea7, State: Running, Role: LEADER
I20260619 22:20:46.880959 1933 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 35323 } }
I20260619 22:20:46.881511 1928 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:20:46.882649 1935 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa63cceb40a44748aa6ed96e5e31ea7. Latest consensus state: current_term: 1 leader_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 35323 } } }
I20260619 22:20:46.882773 1935 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:46.883003 1934 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa63cceb40a44748aa6ed96e5e31ea7" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 35323 } } }
I20260619 22:20:46.883117 1934 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7 [sys.catalog]: This master's current role is: LEADER
W20260619 22:20:46.886544 1949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:20:46.886643 1949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:20:46.886739 1942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:20:46.887470 1942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:20:46.893286 1942 catalog_manager.cc:1347] Generated new cluster ID: a4874ca18fcc45939f30bf27bb2d0561
I20260619 22:20:46.893369 1942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:20:46.900759 1942 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:20:46.901993 1942 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:20:46.907536 1942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfa63cceb40a44748aa6ed96e5e31ea7: Generated new TSK 0
I20260619 22:20:46.908389 1942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:20:46.941886 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:35323
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:47.055508 1954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:47.055828 1954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:47.055874 1954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:47.060218 1954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:47.060305 1954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:47.060375 1954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:20:47.065896 1954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:35323
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:47.067183 1954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:47.068252 1954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:47.078701 1962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:47.078758 1960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:47.078764 1959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:47.078923 1954 server_base.cc:1034] running on GCE node
I20260619 22:20:47.079559 1954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:47.080303 1954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:47.081478 1954 hybrid_clock.cc:648] HybridClock initialized: now 1781907647081457 us; error 36 us; skew 500 ppm
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:47.084641 1954 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:20:47.085572 1954 webserver.cc:460] Webserver started at http://127.1.74.129:36973/ using document root <none> and password file <none>
I20260619 22:20:47.086174 1954 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:47.086237 1954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:47.086483 1954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:47.088716 1954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "65fe68ae19904a94a1cc90591ec962b8"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "fff3d7b34bc4b1d7176441b1adc55a65"
server_key_iv: "7036ff38d3740e1f7b254c6aa9b08d3e"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.089293 1954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "65fe68ae19904a94a1cc90591ec962b8"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "fff3d7b34bc4b1d7176441b1adc55a65"
server_key_iv: "7036ff38d3740e1f7b254c6aa9b08d3e"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.093405 1954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:20:47.095961 1969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.097245 1954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:20:47.097378 1954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "65fe68ae19904a94a1cc90591ec962b8"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "fff3d7b34bc4b1d7176441b1adc55a65"
server_key_iv: "7036ff38d3740e1f7b254c6aa9b08d3e"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.097493 1954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:47.111979 1954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:47.115263 1954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:47.115492 1954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:47.116142 1954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:20:47.117012 1954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:47.117061 1954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.117134 1954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:47.117166 1954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.129480 1954 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:40491
I20260619 22:20:47.129500 2082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:40491 every 8 connection(s)
I20260619 22:20:47.130483 1954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:20:47.138662 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 1954
I20260619 22:20:47.138795 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:20:47.139143 1322 external_mini_cluster.cc:1428] Setting key d5d9fd9961ee9bfd3d4e6b9b87ef704f
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:47.144829 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.132449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:50733 (local address 127.1.74.190:35323)
0619 22:20:47.132591 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.132595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.133480 (+ 885us) server_negotiation.cc:407] Connection header received
0619 22:20:47.134898 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.134902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.134959 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.135042 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.136219 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.136774 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.137928 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.138170 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.138286 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.141088 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.141092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.141094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.141125 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.142641 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.143285 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.143289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.143291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.143374 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.143732 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.143736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.143738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.143926 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.144035 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.144533 (+ 498us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.144666 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:20:47.145699 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.132821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:35323 (local address 127.1.74.129:50733)
0619 22:20:47.133303 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.133347 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.134656 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.135210 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.135217 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.135589 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:47.136070 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.136083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.136910 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.136917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.137796 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.137805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.138328 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.138332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.138554 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.139252 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:47.139275 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:47.140913 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:47.142778 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:47.142784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:47.142802 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:47.143168 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:47.143476 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:47.143479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:47.143481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:47.143629 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:47.144034 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:47.144041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:47.144298 (+ 257us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.144494 (+ 196us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.144845 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
I20260619 22:20:47.147018 2083 heartbeater.cc:347] Connected to a master server at 127.1.74.190:35323
I20260619 22:20:47.147297 2083 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:47.147934 2083 heartbeater.cc:511] Master 127.1.74.190:35323 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:20:47.149684 1892 ts_manager.cc:194] Registered new tserver with Master: 65fe68ae19904a94a1cc90591ec962b8 (127.1.74.129:40491)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:20:47.151301 1892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:50733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:20:47.165910 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:35323
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:47.281900 2088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:47.282258 2088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:47.282351 2088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:47.286676 2088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:47.286751 2088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:47.286885 2088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:20:47.292587 2088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:35323
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:47.293774 2088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:47.294945 2088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:47.305325 2096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:47.305557 2088 server_base.cc:1034] running on GCE node
W20260619 22:20:47.305330 2093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:47.305375 2094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:47.306193 2088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:47.306980 2088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:47.308154 2088 hybrid_clock.cc:648] HybridClock initialized: now 1781907647308132 us; error 35 us; skew 500 ppm
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:47.311188 2088 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:20:47.312243 2088 webserver.cc:460] Webserver started at http://127.1.74.130:37673/ using document root <none> and password file <none>
I20260619 22:20:47.312865 2088 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:47.312956 2088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:47.313198 2088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:47.315505 2088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "89c45d19fb5e443eb199f42b07983f12"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "72f8c97fd6c9d0207c8f84b57de6d3be"
server_key_iv: "af792c4d19770484fa9e9282a36ff0ae"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.316108 2088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "89c45d19fb5e443eb199f42b07983f12"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "72f8c97fd6c9d0207c8f84b57de6d3be"
server_key_iv: "af792c4d19770484fa9e9282a36ff0ae"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.320142 2088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 22:20:47.322772 2103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.324307 2088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:20:47.324435 2088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "89c45d19fb5e443eb199f42b07983f12"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "72f8c97fd6c9d0207c8f84b57de6d3be"
server_key_iv: "af792c4d19770484fa9e9282a36ff0ae"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.324556 2088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:47.340070 2088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:47.343602 2088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:47.343828 2088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:47.344475 2088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:20:47.345396 2088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:47.345444 2088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.345487 2088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:47.345528 2088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.354820 2088 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:38607
I20260619 22:20:47.355001 2216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:38607 every 8 connection(s)
I20260619 22:20:47.355923 2088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:20:47.363087 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 2088
I20260619 22:20:47.363224 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:20:47.363531 1322 external_mini_cluster.cc:1428] Setting key 58d2e355fce3fa0a56a5ae9f57ccf994
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:47.371151 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.357975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:35161 (local address 127.1.74.190:35323)
0619 22:20:47.358121 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.358124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.359087 (+ 963us) server_negotiation.cc:407] Connection header received
0619 22:20:47.360449 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.360451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.360499 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.360582 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.361801 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.362330 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.364094 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.364400 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.364544 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.367226 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.367229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.367231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.367258 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.368739 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.369308 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.369311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.369313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.369391 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.369780 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.369783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.369784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.369969 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.370122 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.370772 (+ 650us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.370954 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":57}
I20260619 22:20:47.372170 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.358391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:35323 (local address 127.1.74.130:35161)
0619 22:20:47.358863 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.358971 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.360236 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.360749 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.360756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.361157 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:47.361643 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.361672 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.362473 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.362476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.363908 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.363922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.364584 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.364588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.364751 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.365470 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:47.365487 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:47.367036 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:47.368870 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:47.368877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:47.368897 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:47.369187 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:47.369495 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:47.369498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:47.369500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:47.369659 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:47.370117 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:47.370124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:47.370494 (+ 370us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.370715 (+ 221us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.371116 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:20:47.373483 2217 heartbeater.cc:347] Connected to a master server at 127.1.74.190:35323
I20260619 22:20:47.373792 2217 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:47.374399 2217 heartbeater.cc:511] Master 127.1.74.190:35323 requested a full tablet report, sending...
I20260619 22:20:47.375555 1892 ts_manager.cc:194] Registered new tserver with Master: 89c45d19fb5e443eb199f42b07983f12 (127.1.74.130:38607)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:20:47.376171 1892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:35161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:20:47.391202 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:35323
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:47.505345 2222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:47.505622 2222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:47.505668 2222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:47.509680 2222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:47.509748 2222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:47.509809 2222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:20:47.515072 2222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:35323
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:47.516116 2222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:47.517174 2222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:20:47.526584 2222 server_base.cc:1034] running on GCE node
W20260619 22:20:47.526528 2230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:47.526501 2227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:47.526503 2228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:47.527258 2222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:47.527949 2222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:47.529160 2222 hybrid_clock.cc:648] HybridClock initialized: now 1781907647529140 us; error 28 us; skew 500 ppm
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:47.531899 2222 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:20:47.532733 2222 webserver.cc:460] Webserver started at http://127.1.74.131:38835/ using document root <none> and password file <none>
I20260619 22:20:47.533291 2222 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:47.533380 2222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:47.533601 2222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:47.535694 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "95d95f39ed47424baa9ae383d3af74d6"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "998d744d97948bb32917e36c9e00929e"
server_key_iv: "af0dacad99212335511f5ed762f8c78e"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.536238 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "95d95f39ed47424baa9ae383d3af74d6"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "998d744d97948bb32917e36c9e00929e"
server_key_iv: "af0dacad99212335511f5ed762f8c78e"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.539928 2222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:20:47.542394 2237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.543706 2222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:20:47.543836 2222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "95d95f39ed47424baa9ae383d3af74d6"
format_stamp: "Formatted at 2026-06-19 22:20:47 on dist-test-slave-k84h"
server_key: "998d744d97948bb32917e36c9e00929e"
server_key_iv: "af0dacad99212335511f5ed762f8c78e"
server_key_version: "encryptionkey@0"
I20260619 22:20:47.543947 2222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:47.558672 2222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:47.562003 2222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:47.562219 2222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:47.562813 2222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:20:47.563669 2222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:47.563738 2222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.563812 2222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:47.563853 2222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:47.572912 2222 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:41081
I20260619 22:20:47.572960 2350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:41081 every 8 connection(s)
I20260619 22:20:47.573870 2222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:20:47.577840 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 2222
I20260619 22:20:47.577961 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:20:47.578231 1322 external_mini_cluster.cc:1428] Setting key b3a75e67bdbea199033dc946b42ab8b4
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:47.588867 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.575886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:55283 (local address 127.1.74.190:35323)
0619 22:20:47.576037 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.576041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.576896 (+ 855us) server_negotiation.cc:407] Connection header received
0619 22:20:47.578625 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.578631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.578702 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.578817 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.580389 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.581023 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.582319 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.582553 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.582650 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.585359 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.585363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.585365 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.585393 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.586695 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.587275 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.587279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.587280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.587387 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.587751 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.587754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.587755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.588002 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.588092 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.588599 (+ 507us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.588719 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:20:47.589715 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.576280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:35323 (local address 127.1.74.131:55283)
0619 22:20:47.576746 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.576789 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.578346 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.579047 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.579055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.579516 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:47.580216 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.580232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.581145 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.581148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.582197 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.582207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.582697 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.582700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.582910 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.583583 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:47.583599 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:47.585174 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:47.586811 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:47.586817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:47.586833 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:47.587159 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:47.587482 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:47.587484 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:47.587486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:47.587627 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:47.588096 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:47.588102 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:47.588363 (+ 261us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.588520 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.588806 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":133,"threads_started":1}
I20260619 22:20:47.591125 2351 heartbeater.cc:347] Connected to a master server at 127.1.74.190:35323
I20260619 22:20:47.591420 2351 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:47.592049 2351 heartbeater.cc:511] Master 127.1.74.190:35323 requested a full tablet report, sending...
I20260619 22:20:47.593231 1892 ts_manager.cc:194] Registered new tserver with Master: 95d95f39ed47424baa9ae383d3af74d6 (127.1.74.131:41081)
I20260619 22:20:47.593976 1892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:55283
I20260619 22:20:47.604744 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:20:47.616398 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.608602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.1.74.190:35323)
0619 22:20:47.608775 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.608781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.608973 (+ 192us) server_negotiation.cc:407] Connection header received
0619 22:20:47.609057 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.609059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.609110 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.609203 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.610061 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.610602 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.611654 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.611878 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.611999 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.613389 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.613392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.613394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.613415 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.614732 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.615170 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.615173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.615174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.615256 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.615495 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.615497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.615499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.615727 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.615831 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.616056 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.616192 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:20:47.621568 1892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:20:47.624703 1892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:47.640236 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.634653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36194 (local address 127.1.74.129:40491)
0619 22:20:47.635060 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.635078 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.635664 (+ 586us) server_negotiation.cc:407] Connection header received
0619 22:20:47.635814 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.635818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.635967 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.636123 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:47.636328 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.637047 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.638607 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.639404 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.639520 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.639674 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.639920 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.640056 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":71,"threads_started":1}
I20260619 22:20:47.640226 2363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.634212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40491 (local address 127.0.0.1:36194)
0619 22:20:47.635580 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.635595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.635681 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.636099 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.636102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.636125 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:47.636236 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.636245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.637219 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.637222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.638468 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.638478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.639559 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.639563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.639694 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.639739 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.639874 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.640018 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":84,"thread_start_us":104,"threads_started":1}
I20260619 22:20:47.640717 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.633746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:41081 (local address 127.0.0.1:49190)
0619 22:20:47.634479 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.634493 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.634575 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.635320 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.635327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.635349 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:47.635501 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.635509 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.637188 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.637191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.639053 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.639065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.640227 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.640231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.640383 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.640409 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.640476 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.640558 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":150,"thread_start_us":134,"threads_started":1}
I20260619 22:20:47.640913 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.633559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:38607 (local address 127.0.0.1:56840)
0619 22:20:47.634046 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.634093 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.634236 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.635719 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.635723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.635743 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:47.635861 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.635868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.637488 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.637491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.639053 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.639065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.640110 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.640114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.640217 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.640631 (+ 414us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.640692 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.640771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":485,"thread_start_us":107,"threads_started":1}
I20260619 22:20:47.641014 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.633744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.1.74.130:38607)
0619 22:20:47.635267 (+ 1523us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.635275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.635295 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:20:47.635359 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.635363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.635522 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.636022 (+ 500us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:47.636174 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.637318 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.639187 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.639973 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.640314 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.640713 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.640765 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.640863 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1441,"thread_start_us":128,"threads_started":1}
I20260619 22:20:47.641470 2366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.634459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.1.74.131:41081)
0619 22:20:47.634816 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.634828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.634874 (+ 46us) server_negotiation.cc:407] Connection header received
0619 22:20:47.634952 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.634959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.635168 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.635332 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:47.635647 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.637005 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.639223 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.640072 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.640982 (+ 910us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.641151 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.641210 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.641307 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":176,"threads_started":1}
I20260619 22:20:47.644022 2017 tablet_service.cc:1467] Processing CreateTablet for tablet 6da144c8f5024913aea13621c4d3f260 (DEFAULT_TABLE table=test-table [id=9cf8b66b994a4cd494fc82127773ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:47.644320 2285 tablet_service.cc:1467] Processing CreateTablet for tablet 6da144c8f5024913aea13621c4d3f260 (DEFAULT_TABLE table=test-table [id=9cf8b66b994a4cd494fc82127773ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:47.644320 2151 tablet_service.cc:1467] Processing CreateTablet for tablet 6da144c8f5024913aea13621c4d3f260 (DEFAULT_TABLE table=test-table [id=9cf8b66b994a4cd494fc82127773ab42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:47.645181 2017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da144c8f5024913aea13621c4d3f260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:47.645442 2285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da144c8f5024913aea13621c4d3f260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:47.645442 2151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da144c8f5024913aea13621c4d3f260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:47.651866 2368 tablet_bootstrap.cc:492] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8: Bootstrap starting.
I20260619 22:20:47.653322 2369 tablet_bootstrap.cc:492] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: Bootstrap starting.
I20260619 22:20:47.654281 2368 tablet_bootstrap.cc:654] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:47.654501 2370 tablet_bootstrap.cc:492] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: Bootstrap starting.
I20260619 22:20:47.655273 2368 log.cc:824] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:47.655507 2369 tablet_bootstrap.cc:654] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:47.656822 2369 log.cc:824] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:47.657238 2370 tablet_bootstrap.cc:654] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:47.657823 2368 tablet_bootstrap.cc:492] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8: No bootstrap required, opened a new log
I20260619 22:20:47.658020 2368 ts_tablet_manager.cc:1397] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:20:47.658164 2370 log.cc:824] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:47.658932 2369 tablet_bootstrap.cc:492] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: No bootstrap required, opened a new log
I20260619 22:20:47.659147 2369 ts_tablet_manager.cc:1397] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:20:47.660172 2370 tablet_bootstrap.cc:492] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: No bootstrap required, opened a new log
I20260619 22:20:47.660393 2370 ts_tablet_manager.cc:1397] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:20:47.663116 2368 raft_consensus.cc:348] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.663422 2368 raft_consensus.cc:374] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:47.663470 2368 raft_consensus.cc:729] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65fe68ae19904a94a1cc90591ec962b8, State: Initialized, Role: FOLLOWER
I20260619 22:20:47.663491 2369 raft_consensus.cc:348] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.663720 2369 raft_consensus.cc:374] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:47.663767 2369 raft_consensus.cc:729] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d95f39ed47424baa9ae383d3af74d6, State: Initialized, Role: FOLLOWER
I20260619 22:20:47.664022 2368 consensus_queue.cc:260] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.663899 2370 raft_consensus.cc:348] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.664187 2370 raft_consensus.cc:374] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:47.664243 2370 raft_consensus.cc:729] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89c45d19fb5e443eb199f42b07983f12, State: Initialized, Role: FOLLOWER
I20260619 22:20:47.664179 2369 consensus_queue.cc:260] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.664705 2370 consensus_queue.cc:260] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.665132 2368 ts_tablet_manager.cc:1428] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 22:20:47.665140 2369 ts_tablet_manager.cc:1428] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:20:47.665503 2351 heartbeater.cc:503] Master 127.1.74.190:35323 was elected leader, sending a full tablet report...
I20260619 22:20:47.665503 2083 heartbeater.cc:503] Master 127.1.74.190:35323 was elected leader, sending a full tablet report...
I20260619 22:20:47.665504 2217 heartbeater.cc:503] Master 127.1.74.190:35323 was elected leader, sending a full tablet report...
I20260619 22:20:47.665760 2370 ts_tablet_manager.cc:1428] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:20:47.757184 2376 raft_consensus.cc:482] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:47.757381 2376 raft_consensus.cc:504] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.758471 2376 leader_election.cc:288] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95d95f39ed47424baa9ae383d3af74d6 (127.1.74.131:41081), 65fe68ae19904a94a1cc90591ec962b8 (127.1.74.129:40491)
I20260619 22:20:47.762287 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.758808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:38781 (local address 127.1.74.129:40491)
0619 22:20:47.758947 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.758952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.758968 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:20:47.759104 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.759109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.759164 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.759250 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:47.759538 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.760169 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.761242 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.761723 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.761813 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.761852 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.762077 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.762146 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:20:47.762287 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.758690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40491 (local address 127.1.74.130:38781)
0619 22:20:47.758839 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.758872 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.758968 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.759269 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.759272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.759290 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:47.759427 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.759434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.760268 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.760272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.761117 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.761125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.761833 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.761837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.761953 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.761981 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.762052 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.762130 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 22:20:47.762440 2366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.758784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:34403 (local address 127.1.74.131:41081)
0619 22:20:47.758920 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.758925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.759387 (+ 462us) server_negotiation.cc:407] Connection header received
0619 22:20:47.759468 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.759474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.759530 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.759596 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:47.759871 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.760410 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.761440 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.761879 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.761996 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.762060 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.762246 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.762323 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 22:20:47.762480 2377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.758690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:41081 (local address 127.1.74.130:34403)
0619 22:20:47.759283 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:47.759296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:47.759392 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:47.759631 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:47.759634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:47.759649 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:47.759750 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.759756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.760517 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.760519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.761317 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:47.761326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.761994 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.761997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:47.762106 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.762137 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 22:20:47.762212 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 22:20:47.762270 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":194,"thread_start_us":140,"threads_started":1}
I20260619 22:20:47.763031 2037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da144c8f5024913aea13621c4d3f260" candidate_uuid: "89c45d19fb5e443eb199f42b07983f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fe68ae19904a94a1cc90591ec962b8" is_pre_election: true
I20260619 22:20:47.763077 2305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da144c8f5024913aea13621c4d3f260" candidate_uuid: "89c45d19fb5e443eb199f42b07983f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d95f39ed47424baa9ae383d3af74d6" is_pre_election: true
I20260619 22:20:47.763342 2305 raft_consensus.cc:2454] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89c45d19fb5e443eb199f42b07983f12 in term 0.
I20260619 22:20:47.763342 2037 raft_consensus.cc:2454] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89c45d19fb5e443eb199f42b07983f12 in term 0.
I20260619 22:20:47.763829 2107 leader_election.cc:302] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 65fe68ae19904a94a1cc90591ec962b8, 89c45d19fb5e443eb199f42b07983f12, 95d95f39ed47424baa9ae383d3af74d6; no voters:
I20260619 22:20:47.764132 2376 raft_consensus.cc:2781] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:47.764204 2376 raft_consensus.cc:482] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:47.764233 2376 raft_consensus.cc:3037] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:47.765266 2376 raft_consensus.cc:504] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.765633 2376 leader_election.cc:288] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [CANDIDATE]: Term 1 election: Requested vote from peers 95d95f39ed47424baa9ae383d3af74d6 (127.1.74.131:41081), 65fe68ae19904a94a1cc90591ec962b8 (127.1.74.129:40491)
I20260619 22:20:47.765990 2037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da144c8f5024913aea13621c4d3f260" candidate_uuid: "89c45d19fb5e443eb199f42b07983f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fe68ae19904a94a1cc90591ec962b8"
I20260619 22:20:47.766013 2305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da144c8f5024913aea13621c4d3f260" candidate_uuid: "89c45d19fb5e443eb199f42b07983f12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d95f39ed47424baa9ae383d3af74d6"
I20260619 22:20:47.766124 2037 raft_consensus.cc:3037] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:47.766113 2305 raft_consensus.cc:3037] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:47.767194 2305 raft_consensus.cc:2454] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89c45d19fb5e443eb199f42b07983f12 in term 1.
I20260619 22:20:47.767292 2037 raft_consensus.cc:2454] T 6da144c8f5024913aea13621c4d3f260 P 65fe68ae19904a94a1cc90591ec962b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89c45d19fb5e443eb199f42b07983f12 in term 1.
I20260619 22:20:47.767522 2107 leader_election.cc:302] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89c45d19fb5e443eb199f42b07983f12, 95d95f39ed47424baa9ae383d3af74d6; no voters:
I20260619 22:20:47.767747 2376 raft_consensus.cc:2781] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:47.767964 2376 raft_consensus.cc:686] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [term 1 LEADER]: Becoming Leader. State: Replica: 89c45d19fb5e443eb199f42b07983f12, State: Running, Role: LEADER
I20260619 22:20:47.768285 2376 consensus_queue.cc:237] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } }
I20260619 22:20:47.771356 1891 catalog_manager.cc:5495] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89c45d19fb5e443eb199f42b07983f12 (127.1.74.130). New cstate: current_term: 1 leader_uuid: "89c45d19fb5e443eb199f42b07983f12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89c45d19fb5e443eb199f42b07983f12" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95d95f39ed47424baa9ae383d3af74d6" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 41081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65fe68ae19904a94a1cc90591ec962b8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40491 } health_report { overall_health: UNKNOWN } } }
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:20:47.796553 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.787179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.1.74.129:40491)
0619 22:20:47.787367 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.787370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.787462 (+ 92us) server_negotiation.cc:407] Connection header received
0619 22:20:47.787568 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.787571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.787620 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.787687 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.788413 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.788945 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.789765 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.790019 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.790114 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.792841 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.792855 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.792872 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.792898 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.794685 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.795206 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.795213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.795214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.795297 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.795592 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.795595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.795597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.795860 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.796077 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.796243 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.796364 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:20:47.823638 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.815655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.1.74.129:40491)
0619 22:20:47.815852 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.815856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.815945 (+ 89us) server_negotiation.cc:407] Connection header received
0619 22:20:47.816021 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.816023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.816070 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.816149 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.816817 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.817386 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.818194 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.818426 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.818511 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.820885 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.820888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.820890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.820915 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.822078 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.822515 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.822517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.822519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.822587 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.822803 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.822805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.822806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.823013 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.823114 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.823373 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.823469 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260619 22:20:47.824928 2352 tablet.cc:2369] T 6da144c8f5024913aea13621c4d3f260 P 95d95f39ed47424baa9ae383d3af74d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:20:47.849022 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:47.841024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.1.74.129:40491)
0619 22:20:47.841196 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:47.841200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:47.841231 (+ 31us) server_negotiation.cc:407] Connection header received
0619 22:20:47.841356 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:47.841359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:47.841400 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:47.841469 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:47.842158 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.842646 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.843523 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:47.843728 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:47.843806 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:47.846013 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:47.846016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:47.846018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:47.846040 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:47.847586 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.847937 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.847939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.847941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.848016 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:47.848241 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:47.848243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:47.848244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:47.848433 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:47.848532 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:47.848739 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 22:20:47.848831 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 22:20:47.850359 1997 tablet_service.cc:2908] Rejecting scan request for tablet 6da144c8f5024913aea13621c4d3f260: Uninitialized: clean time has not yet been initialized
I20260619 22:20:47.852444 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 1954
W20260619 22:20:47.857156 2218 tablet.cc:2369] T 6da144c8f5024913aea13621c4d3f260 P 89c45d19fb5e443eb199f42b07983f12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:20:47.858436 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 2088
I20260619 22:20:47.864080 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 2222
I20260619 22:20:47.870184 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 1860
2026-06-19T22:20:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2401 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[2392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[2392](info): set up 2 sockets
Jun 19 22:20:47 dist-test-slave-k84h krb5kdc[2392](info): commencing operation
krb5kdc: starting...
W20260619 22:20:48.910590 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:20:48 dist-test-slave-k84h krb5kdc[2392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:20:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:20:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:20:49.038782 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:34571
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:34571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:49.154384 2405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:49.154750 2405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:49.154834 2405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:49.159071 2405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:20:49.159205 2405 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 22:20:49.159241 2405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:49.159287 2405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:49.159329 2405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:20:49.159363 2405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:20:49.164960 2405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:34571
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:34571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:49.166193 2405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:49.167358 2405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:49.176206 2410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.176321 2413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.176378 2411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:49.176615 2405 server_base.cc:1034] running on GCE node
I20260619 22:20:49.177211 2405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:49.178372 2405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:49.179536 2405 hybrid_clock.cc:648] HybridClock initialized: now 1781907649179521 us; error 43 us; skew 500 ppm
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:49.185248 2405 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:20:49.186203 2405 webserver.cc:460] Webserver started at http://127.1.74.190:33137/ using document root <none> and password file <none>
I20260619 22:20:49.186794 2405 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:49.186908 2405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:49.187147 2405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:49.189978 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "d6ed84a1f38f48e68738b861f8ad08ce"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "8c68440beafe0f9ce9f5b833652ef0b3"
server_key_iv: "e419e7e5b2a1e614815caea2b125d8f8"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.190644 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "d6ed84a1f38f48e68738b861f8ad08ce"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "8c68440beafe0f9ce9f5b833652ef0b3"
server_key_iv: "e419e7e5b2a1e614815caea2b125d8f8"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.194517 2405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 22:20:49.197149 2420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.198468 2405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:49.198616 2405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "d6ed84a1f38f48e68738b861f8ad08ce"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "8c68440beafe0f9ce9f5b833652ef0b3"
server_key_iv: "e419e7e5b2a1e614815caea2b125d8f8"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.198742 2405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:49.224365 2405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:49.227712 2405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:49.227922 2405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:49.235544 2405 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:34571
I20260619 22:20:49.235567 2482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:34571 every 8 connection(s)
I20260619 22:20:49.236582 2405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:20:49.239799 2483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:49.245630 2483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: Bootstrap starting.
I20260619 22:20:49.246081 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 2405
I20260619 22:20:49.246178 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:20:49.246445 1322 external_mini_cluster.cc:1428] Setting key a6426e21c0d425b6c3df92194f04da99
I20260619 22:20:49.248412 2483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:49.249315 2483 log.cc:824] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:49.251299 2483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: No bootstrap required, opened a new log
I20260619 22:20:49.255384 2483 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 34571 } }
I20260619 22:20:49.255645 2483 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:49.255725 2483 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ed84a1f38f48e68738b861f8ad08ce, State: Initialized, Role: FOLLOWER
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:49.256153 2483 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 34571 } }
I20260619 22:20:49.256314 2483 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:20:49.256404 2483 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:20:49.256520 2483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:49.257556 2483 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 34571 } }
I20260619 22:20:49.257903 2483 leader_election.cc:302] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6ed84a1f38f48e68738b861f8ad08ce; no voters:
I20260619 22:20:49.258232 2483 leader_election.cc:288] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:20:49.258350 2488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:49.258543 2488 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [term 1 LEADER]: Becoming Leader. State: Replica: d6ed84a1f38f48e68738b861f8ad08ce, State: Running, Role: LEADER
I20260619 22:20:49.258884 2488 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 34571 } }
I20260619 22:20:49.259490 2483 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:20:49.260569 2489 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ed84a1f38f48e68738b861f8ad08ce. Latest consensus state: current_term: 1 leader_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 34571 } } }
I20260619 22:20:49.260699 2489 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:49.260955 2490 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ed84a1f38f48e68738b861f8ad08ce" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 34571 } } }
I20260619 22:20:49.261058 2490 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:49.261344 2497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:20:49.262974 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.247843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.1.74.190:34571)
0619 22:20:49.248266 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.248287 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.248333 (+ 46us) server_negotiation.cc:407] Connection header received
0619 22:20:49.249257 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.249281 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.249738 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.250105 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.250977 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.252444 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.253352 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.253857 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.254037 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.256890 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.256896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.256921 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.256946 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.259015 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.259560 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.259566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.259571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.259699 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.260049 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.260053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.260054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.260593 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.260814 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.261718 (+ 904us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.262003 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":114,"threads_started":1}
I20260619 22:20:49.265139 2497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:20:49.270386 2497 catalog_manager.cc:1347] Generated new cluster ID: 663ede04e99e44f8b82f011a80e66fff
I20260619 22:20:49.270449 2497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:20:49.286273 2497 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:20:49.287508 2497 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:20:49.294797 2497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: Generated new TSK 0
I20260619 22:20:49.295519 2497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:20:49.329690 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:34571
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:49.441179 2509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:49.441428 2509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:49.441475 2509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:49.445465 2509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:49.445523 2509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:49.445561 2509 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:20:49.445597 2509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:20:49.450686 2509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:34571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:49.451699 2509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:49.452657 2509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:49.462779 2515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.462740 2517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.462790 2514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:49.463088 2509 server_base.cc:1034] running on GCE node
I20260619 22:20:49.463505 2509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:49.464231 2509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:49.465452 2509 hybrid_clock.cc:648] HybridClock initialized: now 1781907649465416 us; error 48 us; skew 500 ppm
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:49.468497 2509 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:20:49.469416 2509 webserver.cc:460] Webserver started at http://127.1.74.129:34177/ using document root <none> and password file <none>
I20260619 22:20:49.470005 2509 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:49.470108 2509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:49.470342 2509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:49.472504 2509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "6fcb415f4c784660b69dd48d559ae3c4"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "c917de1df580286dd4e080c73cc0d547"
server_key_iv: "a87c164e856212113876f12f7132fe3d"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.473065 2509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "6fcb415f4c784660b69dd48d559ae3c4"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "c917de1df580286dd4e080c73cc0d547"
server_key_iv: "a87c164e856212113876f12f7132fe3d"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.476976 2509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:20:49.479679 2524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.481019 2509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:20:49.481201 2509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "6fcb415f4c784660b69dd48d559ae3c4"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "c917de1df580286dd4e080c73cc0d547"
server_key_iv: "a87c164e856212113876f12f7132fe3d"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.481324 2509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:49.510097 2509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:49.513631 2509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:49.513859 2509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:49.515475 2509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:49.515552 2509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.515623 2509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:49.515666 2509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:49.526783 2509 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:37853
I20260619 22:20:49.526860 2639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:37853 every 8 connection(s)
I20260619 22:20:49.527954 2509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:20:49.530539 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.516343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:46313 (local address 127.1.74.190:34571)
0619 22:20:49.516522 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.516530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.517531 (+ 1001us) server_negotiation.cc:407] Connection header received
0619 22:20:49.518872 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.518879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.518951 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.519022 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.520952 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.521483 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.523440 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.523786 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.523880 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.526740 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.526744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.526746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.526779 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.528341 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.528953 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.528956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.528958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.529052 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.529388 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.529393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.529395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.529608 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.529714 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.530203 (+ 489us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.530332 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 22:20:49.531379 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.516779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.1.74.129:46313)
0619 22:20:49.517338 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:49.517401 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:49.518554 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:49.519484 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:49.519501 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:49.520022 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:49.520790 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.520817 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.521604 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.521609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.523298 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.523314 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.523954 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.523959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.524192 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.524947 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:49.524968 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:49.526514 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:49.528494 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.528499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.528514 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.528847 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.529148 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.529151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.529154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.529293 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.529712 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:49.529719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:49.529973 (+ 254us) client_negotiation.cc:769] Sending connection context
0619 22:20:49.530160 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 22:20:49.530477 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
I20260619 22:20:49.532773 2640 heartbeater.cc:347] Connected to a master server at 127.1.74.190:34571
I20260619 22:20:49.532938 2640 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:49.533499 2640 heartbeater.cc:511] Master 127.1.74.190:34571 requested a full tablet report, sending...
I20260619 22:20:49.535295 2436 ts_manager.cc:194] Registered new tserver with Master: 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853)
I20260619 22:20:49.536487 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 2509
I20260619 22:20:49.536577 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:20:49.536849 1322 external_mini_cluster.cc:1428] Setting key e33df437dfaa0247fecaaaed16eaff6d
I20260619 22:20:49.536798 2436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:46313
I20260619 22:20:49.542786 2646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.534883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.1.74.129:50291)
0619 22:20:49.535180 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:49.535195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:49.535304 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:49.535548 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:49.535551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:49.535771 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:49.535894 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.535901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.536774 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.536777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.537493 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.537502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.538165 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.538168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.538268 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.538901 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:49.538915 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:49.539367 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:49.541236 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.541245 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.541246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.541442 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.541754 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.541756 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.541758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.541831 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.542319 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:49.542322 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:49.542405 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 22:20:49.542506 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 22:20:49.542625 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":125,"threads_started":1}
I20260619 22:20:49.543013 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.534954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:50291 (local address 127.1.74.190:34571)
0619 22:20:49.535088 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.535093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.535277 (+ 184us) server_negotiation.cc:407] Connection header received
0619 22:20:49.535407 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.535409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.535448 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.535507 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.535999 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.536649 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.537717 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.538045 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.538147 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.539520 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.539526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.539533 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.539565 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.541129 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.541538 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.541545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.541548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.541643 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.541919 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.541925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.541930 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.542192 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.542303 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.542600 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.542748 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:20:49.566049 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:34571
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:49.680153 2649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:49.680441 2649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:49.680487 2649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:49.684754 2649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:49.684825 2649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:49.684867 2649 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:20:49.684901 2649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:20:49.690441 2649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:34571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:49.691663 2649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:49.692735 2649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:49.702737 2655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.702737 2657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.702818 2654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:49.702943 2649 server_base.cc:1034] running on GCE node
I20260619 22:20:49.703438 2649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:49.704164 2649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:49.705354 2649 hybrid_clock.cc:648] HybridClock initialized: now 1781907649705320 us; error 51 us; skew 500 ppm
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:49.708261 2649 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:20:49.709184 2649 webserver.cc:460] Webserver started at http://127.1.74.130:43775/ using document root <none> and password file <none>
I20260619 22:20:49.709796 2649 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:49.709892 2649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:49.710127 2649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:49.712355 2649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "25db844447c94f579ea2087f997f1285"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "89b5ee4578ebf09928737c1e8a43e96e"
server_key_iv: "f8046af94ef375d9a8b84ae6885b4a32"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.712929 2649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "25db844447c94f579ea2087f997f1285"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "89b5ee4578ebf09928737c1e8a43e96e"
server_key_iv: "f8046af94ef375d9a8b84ae6885b4a32"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.716737 2649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 22:20:49.719221 2664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.720445 2649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:20:49.720582 2649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "25db844447c94f579ea2087f997f1285"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "89b5ee4578ebf09928737c1e8a43e96e"
server_key_iv: "f8046af94ef375d9a8b84ae6885b4a32"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.720706 2649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:49.743641 2649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:49.746975 2649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:49.747195 2649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:49.748700 2649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:49.748778 2649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.748854 2649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:49.748898 2649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:49.759953 2649 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:34381
I20260619 22:20:49.760005 2779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:34381 every 8 connection(s)
I20260619 22:20:49.761500 2649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:20:49.763168 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 2649
I20260619 22:20:49.763247 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.749731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:52183 (local address 127.1.74.190:34571)
0619 22:20:49.749883 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.749888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.750884 (+ 996us) server_negotiation.cc:407] Connection header received
0619 22:20:49.752175 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.752195 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.752270 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.752373 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.753643 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.754178 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.755417 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.755690 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.755794 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.758408 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.758414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.758422 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.758454 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.760257 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.760795 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.760799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.760800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.760900 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.761355 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.761359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.761360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.761632 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.761751 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.762990 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.763110 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:20:49.763288 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:20:49.763537 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.750221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.1.74.130:52183)
0619 22:20:49.750699 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:49.750751 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:49.751907 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:49.752601 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:49.752608 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:49.752997 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:49.753491 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.753505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.754305 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.754309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.755292 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.755301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.755837 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.755841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.756038 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.756722 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:49.756742 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:49.758231 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:49.760368 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.760372 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.760384 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.760696 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.761028 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.761032 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.761034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.761208 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.761751 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:49.761760 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:49.762117 (+ 357us) client_negotiation.cc:769] Sending connection context
0619 22:20:49.762320 (+ 203us) client_negotiation.cc:240] Negotiation successful
0619 22:20:49.762600 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":134,"threads_started":1}
I20260619 22:20:49.763563 1322 external_mini_cluster.cc:1428] Setting key a39fc46f52c1dab302595634a069c344
I20260619 22:20:49.764798 2780 heartbeater.cc:347] Connected to a master server at 127.1.74.190:34571
I20260619 22:20:49.764951 2780 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:49.765507 2780 heartbeater.cc:511] Master 127.1.74.190:34571 requested a full tablet report, sending...
I20260619 22:20:49.766510 2435 ts_manager.cc:194] Registered new tserver with Master: 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381)
I20260619 22:20:49.767316 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:52183
I20260619 22:20:49.775393 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.766936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.1.74.130:37303)
0619 22:20:49.767210 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:49.767225 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:49.767324 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:49.767687 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:49.767692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:49.767931 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:49.768082 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.768093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.769093 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.769099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.770043 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.770053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.770699 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.770702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.770773 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.771435 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:49.771453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:49.771786 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:49.773792 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.773794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.773796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.773989 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.774297 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.774299 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.774301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.774375 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.774949 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:49.774951 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:49.775026 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 22:20:49.775137 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 22:20:49.775245 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260619 22:20:49.775619 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.766791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:37303 (local address 127.1.74.190:34571)
0619 22:20:49.766941 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.766946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.767313 (+ 367us) server_negotiation.cc:407] Connection header received
0619 22:20:49.767413 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.767416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.767481 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.767559 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.768220 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.768947 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.770160 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.770562 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.770696 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.771937 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.771944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.771950 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.771983 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.773665 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.774099 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.774102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.774104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.774202 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.774469 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.774473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.774474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.774791 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.774953 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.775228 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.775400 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:20:49.794615 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:34571
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:49.909677 2789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:49.910038 2789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:49.910135 2789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:49.914453 2789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:49.914567 2789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:49.914644 2789 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:20:49.914731 2789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:20:49.920358 2789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:34571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:49.921522 2789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:49.922612 2789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:49.932909 2797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:49.933027 2789 server_base.cc:1034] running on GCE node
W20260619 22:20:49.932920 2794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:49.932911 2795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:49.933661 2789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:49.934402 2789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:49.935583 2789 hybrid_clock.cc:648] HybridClock initialized: now 1781907649935566 us; error 35 us; skew 500 ppm
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:49.938488 2789 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:20:49.939378 2789 webserver.cc:460] Webserver started at http://127.1.74.131:39663/ using document root <none> and password file <none>
I20260619 22:20:49.939906 2789 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:49.939962 2789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:49.940124 2789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:49.942162 2789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "800e2687297c4adcbb2cd149ecf90d29"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "a0136869eaddf39b653fc1536badbf6a"
server_key_iv: "11de407c66d1e6ee28147bdb04192daf"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.942744 2789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "800e2687297c4adcbb2cd149ecf90d29"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "a0136869eaddf39b653fc1536badbf6a"
server_key_iv: "11de407c66d1e6ee28147bdb04192daf"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.946749 2789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:20:49.949528 2804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.951128 2789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 22:20:49.951267 2789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "800e2687297c4adcbb2cd149ecf90d29"
format_stamp: "Formatted at 2026-06-19 22:20:49 on dist-test-slave-k84h"
server_key: "a0136869eaddf39b653fc1536badbf6a"
server_key_iv: "11de407c66d1e6ee28147bdb04192daf"
server_key_version: "encryptionkey@0"
I20260619 22:20:49.951390 2789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:49.966615 2789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:49.969565 2789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:49.969772 2789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:49.971180 2789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:49.971261 2789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:49.971329 2789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:49.971361 2789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:20:49 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:49.981840 2789 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:37915
I20260619 22:20:49.981914 2919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:37915 every 8 connection(s)
I20260619 22:20:49.982971 2789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:20:49.985653 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.972052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49801 (local address 127.1.74.190:34571)
0619 22:20:49.972224 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.972231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.973021 (+ 790us) server_negotiation.cc:407] Connection header received
0619 22:20:49.974224 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.974231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.974299 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.974404 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.975698 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.976661 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.977952 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.978195 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.978307 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.981901 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.981906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.981908 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.981935 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.983548 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.984102 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.984107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.984109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.984201 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.984534 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.984538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.984540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.984721 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.984830 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.985374 (+ 544us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.985486 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:20:49.986383 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.972411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.1.74.131:49801)
0619 22:20:49.972865 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:49.972912 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:49.973976 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:49.974631 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:49.974643 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:49.975069 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:49.975570 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.975582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.976792 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.976796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.977842 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.977856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.978354 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.978358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.978528 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.979928 (+ 1400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:49.979951 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:49.981735 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:49.983671 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.983678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.983693 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.983992 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.984305 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.984308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.984310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.984441 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.984840 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:49.984848 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:49.985141 (+ 293us) client_negotiation.cc:769] Sending connection context
0619 22:20:49.985319 (+ 178us) client_negotiation.cc:240] Negotiation successful
0619 22:20:49.985611 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260619 22:20:49.987581 2920 heartbeater.cc:347] Connected to a master server at 127.1.74.190:34571
I20260619 22:20:49.987715 2920 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:49.988286 2920 heartbeater.cc:511] Master 127.1.74.190:34571 requested a full tablet report, sending...
I20260619 22:20:49.989337 2435 ts_manager.cc:194] Registered new tserver with Master: 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915)
I20260619 22:20:49.990082 2435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:49801
I20260619 22:20:49.991873 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 2789
I20260619 22:20:49.991997 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:20:49.992273 1322 external_mini_cluster.cc:1428] Setting key 8a394243c0f7d9b14f15eb7941879540
I20260619 22:20:49.994685 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:20:49.997565 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.989949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:56627 (local address 127.1.74.190:34571)
0619 22:20:49.990065 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:49.990069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:49.990195 (+ 126us) server_negotiation.cc:407] Connection header received
0619 22:20:49.990335 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:49.990338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:49.990405 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:49.990484 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:49.990968 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.991474 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.992472 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.992763 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.992862 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.994084 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:49.994087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:49.994089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:49.994110 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:49.995823 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.996221 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.996228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.996232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.996322 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:49.996632 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:49.996637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:49.996641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:49.996883 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:49.996991 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:49.997347 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 22:20:49.997449 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 22:20:49.997711 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:49.989805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.1.74.131:56627)
0619 22:20:49.990104 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:49.990124 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:49.990236 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:49.990497 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:49.990499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:49.990696 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:49.990814 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.990821 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.991617 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.991620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.992335 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:49.992362 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:49.992899 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:49.992902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:49.992994 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:49.993533 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:49.993546 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:49.993960 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:49.995944 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.995946 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.995948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.996130 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.996437 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:49.996439 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:49.996441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:49.996535 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:49.997032 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:49.997036 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:49.997157 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 22:20:49.997291 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 22:20:49.997460 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[2392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:50.021317 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.013152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.1.74.190:34571)
0619 22:20:50.013289 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.013292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.013306 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:20:50.013350 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.013352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.013393 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.013473 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:50.014199 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.014722 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.015581 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.015792 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.015874 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.018268 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:50.018271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:50.018273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:50.018297 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:50.019807 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.020203 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.020206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.020207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.020292 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.020521 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.020523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.020524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.020714 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:50.020831 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.021065 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.021172 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:20:50.024703 2435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:20:50.027866 2435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:50.042343 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.036547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:37853 (local address 127.0.0.1:56000)
0619 22:20:50.037326 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.037341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.037433 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.038168 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.038171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.038184 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.038299 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.038307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.039483 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.039487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.040879 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.040889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.041787 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.041790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.041893 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.041927 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.042073 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.042193 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":34,"thread_start_us":99,"threads_started":1}
I20260619 22:20:50.042351 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.037158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.1.74.129:37853)
0619 22:20:50.037684 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.037690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.037713 (+ 23us) server_negotiation.cc:407] Connection header received
0619 22:20:50.037785 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.037792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.038039 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.038191 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.038429 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.039372 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.041006 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.041658 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.041790 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.041939 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.042108 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.042206 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":82,"threads_started":1}
I20260619 22:20:50.042779 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.036478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34381 (local address 127.0.0.1:56702)
0619 22:20:50.037081 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.037131 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.037276 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.038054 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.038060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.038089 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.038334 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.038343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.039816 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.039819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.041065 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.041072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.042326 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.042330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.042483 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.042517 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.042595 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.042655 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":97,"thread_start_us":85,"threads_started":1}
I20260619 22:20:50.042886 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.036588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.1.74.130:34381)
0619 22:20:50.037034 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.037047 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.037424 (+ 377us) server_negotiation.cc:407] Connection header received
0619 22:20:50.037509 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.037515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.037757 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.037913 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.038454 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.039699 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.041170 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.042009 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.042118 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.042261 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.042632 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.042725 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":131,"threads_started":1}
I20260619 22:20:50.043432 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.037208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37915 (local address 127.0.0.1:58170)
0619 22:20:50.038113 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.038127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.038217 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.038682 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.038685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.038704 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.038777 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.038783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.040020 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.040026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.041137 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.041147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.042326 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.042330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.043138 (+ 808us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.043168 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.043240 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.043315 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":362,"spinlock_wait_cycles":1920,"thread_start_us":81,"threads_started":1}
I20260619 22:20:50.043898 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.037586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.1.74.131:37915)
0619 22:20:50.037955 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.037968 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.038279 (+ 311us) server_negotiation.cc:407] Connection header received
0619 22:20:50.038357 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.038365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.038547 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.038685 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.038905 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.039920 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.041253 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.042004 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.042112 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.042261 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.043675 (+ 1414us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.043769 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":98,"threads_started":1}
I20260619 22:20:50.045300 2573 tablet_service.cc:1467] Processing CreateTablet for tablet dd018c3e469744d8822cf1479dd20117 (DEFAULT_TABLE table=test-table [id=8649d426ae81430896157a2721060ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:50.045420 2714 tablet_service.cc:1467] Processing CreateTablet for tablet dd018c3e469744d8822cf1479dd20117 (DEFAULT_TABLE table=test-table [id=8649d426ae81430896157a2721060ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:50.046339 2573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd018c3e469744d8822cf1479dd20117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.046343 2714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd018c3e469744d8822cf1479dd20117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.048318 2854 tablet_service.cc:1467] Processing CreateTablet for tablet dd018c3e469744d8822cf1479dd20117 (DEFAULT_TABLE table=test-table [id=8649d426ae81430896157a2721060ecb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:50.049352 2854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd018c3e469744d8822cf1479dd20117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.052755 2943 tablet_bootstrap.cc:492] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Bootstrap starting.
I20260619 22:20:50.054112 2944 tablet_bootstrap.cc:492] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Bootstrap starting.
I20260619 22:20:50.055768 2943 tablet_bootstrap.cc:654] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.056283 2944 tablet_bootstrap.cc:654] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.056556 2943 log.cc:824] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:50.057075 2944 log.cc:824] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:50.057225 2945 tablet_bootstrap.cc:492] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Bootstrap starting.
I20260619 22:20:50.058399 2943 tablet_bootstrap.cc:492] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: No bootstrap required, opened a new log
I20260619 22:20:50.058597 2943 ts_tablet_manager.cc:1397] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:20:50.058662 2944 tablet_bootstrap.cc:492] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: No bootstrap required, opened a new log
I20260619 22:20:50.058908 2944 ts_tablet_manager.cc:1397] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260619 22:20:50.062114 2943 raft_consensus.cc:348] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.062364 2943 raft_consensus.cc:374] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.062448 2943 raft_consensus.cc:729] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25db844447c94f579ea2087f997f1285, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.062320 2944 raft_consensus.cc:348] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.062637 2944 raft_consensus.cc:374] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.062728 2944 raft_consensus.cc:729] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcb415f4c784660b69dd48d559ae3c4, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.062966 2943 consensus_queue.cc:260] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.063210 2944 consensus_queue.cc:260] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.063685 2780 heartbeater.cc:503] Master 127.1.74.190:34571 was elected leader, sending a full tablet report...
I20260619 22:20:50.063813 2640 heartbeater.cc:503] Master 127.1.74.190:34571 was elected leader, sending a full tablet report...
I20260619 22:20:50.064015 2943 ts_tablet_manager.cc:1428] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:20:50.065472 2944 ts_tablet_manager.cc:1428] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260619 22:20:50.067744 2945 tablet_bootstrap.cc:654] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.068471 2945 log.cc:824] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:50.070327 2945 tablet_bootstrap.cc:492] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: No bootstrap required, opened a new log
I20260619 22:20:50.070531 2945 ts_tablet_manager.cc:1397] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260619 22:20:50.073570 2945 raft_consensus.cc:348] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.073801 2945 raft_consensus.cc:374] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.073864 2945 raft_consensus.cc:729] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800e2687297c4adcbb2cd149ecf90d29, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.074229 2945 consensus_queue.cc:260] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.074820 2920 heartbeater.cc:503] Master 127.1.74.190:34571 was elected leader, sending a full tablet report...
I20260619 22:20:50.075138 2945 ts_tablet_manager.cc:1428] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:20:50.080518 2951 raft_consensus.cc:482] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:50.080693 2951 raft_consensus.cc:504] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.081712 2951 leader_election.cc:288] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381), 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853)
I20260619 22:20:50.086294 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.082081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:42723 (local address 127.1.74.129:37853)
0619 22:20:50.082191 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.082195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.082209 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:20:50.082282 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.082286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.082339 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.082421 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.082743 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.083568 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.085002 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.085675 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.085791 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.085865 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.086076 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.086139 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 22:20:50.086301 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.081971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:37853 (local address 127.1.74.131:42723)
0619 22:20:50.082086 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.082105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.082188 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.082459 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.082462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.082479 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.082603 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.082608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.083683 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.083686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.084882 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.084890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.085801 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.085804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.085926 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.085970 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.086050 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.086151 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 22:20:50.087090 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd018c3e469744d8822cf1479dd20117" candidate_uuid: "800e2687297c4adcbb2cd149ecf90d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcb415f4c784660b69dd48d559ae3c4" is_pre_election: true
I20260619 22:20:50.087231 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.082182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:33831 (local address 127.1.74.130:34381)
0619 22:20:50.082314 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.082318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.082687 (+ 369us) server_negotiation.cc:407] Connection header received
0619 22:20:50.082924 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.082930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.082979 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.083060 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.083683 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.084478 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.085994 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.086605 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.086713 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.086769 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.087026 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.087110 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:20:50.087445 2594 raft_consensus.cc:2454] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 800e2687297c4adcbb2cd149ecf90d29 in term 0.
I20260619 22:20:50.087567 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.081971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34381 (local address 127.1.74.131:33831)
0619 22:20:50.082591 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.082602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.082708 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.083089 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.083092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.083109 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.083546 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.083555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.084601 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.084603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.085866 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.085877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.086700 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.086702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.086813 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.086835 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.087354 (+ 519us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.087423 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":296,"thread_start_us":148,"threads_started":1}
I20260619 22:20:50.087930 2806 leader_election.cc:302] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fcb415f4c784660b69dd48d559ae3c4, 800e2687297c4adcbb2cd149ecf90d29; no voters:
I20260619 22:20:50.088233 2951 raft_consensus.cc:2781] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:50.088219 2734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd018c3e469744d8822cf1479dd20117" candidate_uuid: "800e2687297c4adcbb2cd149ecf90d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25db844447c94f579ea2087f997f1285" is_pre_election: true
I20260619 22:20:50.088323 2951 raft_consensus.cc:482] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:50.088368 2951 raft_consensus.cc:3037] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.088482 2734 raft_consensus.cc:2454] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 800e2687297c4adcbb2cd149ecf90d29 in term 0.
I20260619 22:20:50.089509 2951 raft_consensus.cc:504] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.089862 2951 leader_election.cc:288] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [CANDIDATE]: Term 1 election: Requested vote from peers 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381), 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853)
I20260619 22:20:50.090158 2734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd018c3e469744d8822cf1479dd20117" candidate_uuid: "800e2687297c4adcbb2cd149ecf90d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25db844447c94f579ea2087f997f1285"
I20260619 22:20:50.090301 2734 raft_consensus.cc:3037] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.090590 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd018c3e469744d8822cf1479dd20117" candidate_uuid: "800e2687297c4adcbb2cd149ecf90d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcb415f4c784660b69dd48d559ae3c4"
I20260619 22:20:50.090718 2594 raft_consensus.cc:3037] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.091650 2734 raft_consensus.cc:2454] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800e2687297c4adcbb2cd149ecf90d29 in term 1.
I20260619 22:20:50.091960 2805 leader_election.cc:302] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25db844447c94f579ea2087f997f1285, 800e2687297c4adcbb2cd149ecf90d29; no voters:
I20260619 22:20:50.092015 2594 raft_consensus.cc:2454] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 800e2687297c4adcbb2cd149ecf90d29 in term 1.
I20260619 22:20:50.092133 2951 raft_consensus.cc:2781] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:50.092437 2951 raft_consensus.cc:686] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 LEADER]: Becoming Leader. State: Replica: 800e2687297c4adcbb2cd149ecf90d29, State: Running, Role: LEADER
I20260619 22:20:50.092760 2951 consensus_queue.cc:237] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.095577 2435 catalog_manager.cc:5495] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131). New cstate: current_term: 1 leader_uuid: "800e2687297c4adcbb2cd149ecf90d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } health_report { overall_health: HEALTHY } } }
I20260619 22:20:50.117406 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.113620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.1.74.131:37915)
0619 22:20:50.113762 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.113766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.113781 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:20:50.114039 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.114042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.114100 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.114193 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:50.114492 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.115194 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.116023 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.116254 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.116344 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.116616 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.117055 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.117147 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.117231 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.117283 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:20:50.123407 2594 raft_consensus.cc:1261] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Refusing update from remote peer 800e2687297c4adcbb2cd149ecf90d29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:50.123405 2734 raft_consensus.cc:1261] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Refusing update from remote peer 800e2687297c4adcbb2cd149ecf90d29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:50.124071 2951 consensus_queue.cc:1035] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:50.124305 2953 consensus_queue.cc:1035] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:50.133055 2957 mvcc.cc:205] Tried to move back new op lower bound from 7298693734898450432 to 7298693734782697472. Current Snapshot: MvccSnapshot[applied={T|T < 7298693734898450432}]
I20260619 22:20:50.133769 2958 mvcc.cc:205] Tried to move back new op lower bound from 7298693734898450432 to 7298693734782697472. Current Snapshot: MvccSnapshot[applied={T|T < 7298693734898450432}]
I20260619 22:20:50.144342 2436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:50.144537 2436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:50.146958 2436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: Sending DeleteTablet for 3 replicas of tablet dd018c3e469744d8822cf1479dd20117
I20260619 22:20:50.147795 2714 tablet_service.cc:1514] Processing DeleteTablet for tablet dd018c3e469744d8822cf1479dd20117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:50 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56702
I20260619 22:20:50.147787 2854 tablet_service.cc:1514] Processing DeleteTablet for tablet dd018c3e469744d8822cf1479dd20117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:50 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58170
I20260619 22:20:50.147796 2573 tablet_service.cc:1514] Processing DeleteTablet for tablet dd018c3e469744d8822cf1479dd20117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:50 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56000
I20260619 22:20:50.148695 2967 tablet_replica.cc:330] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: stopping tablet replica
I20260619 22:20:50.148904 2966 tablet_replica.cc:330] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: stopping tablet replica
I20260619 22:20:50.149032 2967 raft_consensus.cc:2229] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:50.149044 2436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 22:20:50.149384 2967 raft_consensus.cc:2258] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 22:20:50.149480 2436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:50.149515 2966 raft_consensus.cc:2229] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:20:50.149891 2968 tablet_replica.cc:330] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: stopping tablet replica
I20260619 22:20:50.150110 2966 raft_consensus.cc:2258] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:50.150177 2968 raft_consensus.cc:2229] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:50.150444 2968 raft_consensus.cc:2258] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:50.151645 2967 ts_tablet_manager.cc:1905] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:50.152470 2966 ts_tablet_manager.cc:1905] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:50.154366 2967 ts_tablet_manager.cc:1918] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:20:50.154464 2967 log.cc:1196] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/dd018c3e469744d8822cf1479dd20117
I20260619 22:20:50.155138 2714 tablet_service.cc:1467] Processing CreateTablet for tablet c99ba9b3cd1949e69079eb06e8500734 (DEFAULT_TABLE table=test-table [id=8ee81a2d5b884a88ba93f83f3162c372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:50.155210 2573 tablet_service.cc:1467] Processing CreateTablet for tablet c99ba9b3cd1949e69079eb06e8500734 (DEFAULT_TABLE table=test-table [id=8ee81a2d5b884a88ba93f83f3162c372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:50.155318 2854 tablet_service.cc:1467] Processing CreateTablet for tablet c99ba9b3cd1949e69079eb06e8500734 (DEFAULT_TABLE table=test-table [id=8ee81a2d5b884a88ba93f83f3162c372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:50.155427 2714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ba9b3cd1949e69079eb06e8500734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.155484 2573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ba9b3cd1949e69079eb06e8500734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.155556 2854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c99ba9b3cd1949e69079eb06e8500734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.157752 2943 tablet_bootstrap.cc:492] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Bootstrap starting.
I20260619 22:20:50.158004 2967 ts_tablet_manager.cc:1939] T dd018c3e469744d8822cf1479dd20117 P 25db844447c94f579ea2087f997f1285: Deleting consensus metadata
I20260619 22:20:50.158205 2944 tablet_bootstrap.cc:492] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Bootstrap starting.
I20260619 22:20:50.158779 2943 tablet_bootstrap.cc:654] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.159389 2945 tablet_bootstrap.cc:492] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Bootstrap starting.
I20260619 22:20:50.159399 2421 catalog_manager.cc:4841] TS 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381): tablet dd018c3e469744d8822cf1479dd20117 (table test-table [id=8649d426ae81430896157a2721060ecb]) successfully deleted
I20260619 22:20:50.159898 2944 tablet_bootstrap.cc:654] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.160341 2945 tablet_bootstrap.cc:654] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.161479 2945 tablet_bootstrap.cc:492] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: No bootstrap required, opened a new log
I20260619 22:20:50.161564 2945 ts_tablet_manager.cc:1397] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:50.161586 2968 ts_tablet_manager.cc:1905] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:50.162020 2945 raft_consensus.cc:348] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.162137 2945 raft_consensus.cc:374] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.162170 2945 raft_consensus.cc:729] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800e2687297c4adcbb2cd149ecf90d29, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.162288 2945 consensus_queue.cc:260] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.162540 2945 ts_tablet_manager.cc:1428] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:50.164316 2968 ts_tablet_manager.cc:1918] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:20:50.164420 2968 log.cc:1196] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/dd018c3e469744d8822cf1479dd20117
I20260619 22:20:50.164749 2968 ts_tablet_manager.cc:1939] T dd018c3e469744d8822cf1479dd20117 P 6fcb415f4c784660b69dd48d559ae3c4: Deleting consensus metadata
I20260619 22:20:50.165257 2966 ts_tablet_manager.cc:1918] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:20:50.165356 2966 log.cc:1196] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/dd018c3e469744d8822cf1479dd20117
I20260619 22:20:50.165663 2966 ts_tablet_manager.cc:1939] T dd018c3e469744d8822cf1479dd20117 P 800e2687297c4adcbb2cd149ecf90d29: Deleting consensus metadata
I20260619 22:20:50.166100 2422 catalog_manager.cc:4841] TS 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853): tablet dd018c3e469744d8822cf1479dd20117 (table test-table [id=8649d426ae81430896157a2721060ecb]) successfully deleted
I20260619 22:20:50.166700 2421 catalog_manager.cc:4841] TS 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915): tablet dd018c3e469744d8822cf1479dd20117 (table test-table [id=8649d426ae81430896157a2721060ecb]) successfully deleted
I20260619 22:20:50.167311 2943 tablet_bootstrap.cc:492] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: No bootstrap required, opened a new log
I20260619 22:20:50.167420 2943 ts_tablet_manager.cc:1397] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260619 22:20:50.167516 2944 tablet_bootstrap.cc:492] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: No bootstrap required, opened a new log
I20260619 22:20:50.167618 2944 ts_tablet_manager.cc:1397] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260619 22:20:50.167965 2943 raft_consensus.cc:348] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.168110 2943 raft_consensus.cc:374] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.168152 2943 raft_consensus.cc:729] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25db844447c94f579ea2087f997f1285, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.168300 2943 consensus_queue.cc:260] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.168311 2944 raft_consensus.cc:348] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.168408 2944 raft_consensus.cc:374] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.168447 2944 raft_consensus.cc:729] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcb415f4c784660b69dd48d559ae3c4, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.168589 2943 ts_tablet_manager.cc:1428] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:50.168570 2944 consensus_queue.cc:260] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.168835 2944 ts_tablet_manager.cc:1428] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260619 22:20:50.233861 2921 tablet.cc:2369] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:20:50.262825 2781 tablet.cc:2369] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:20:50.268641 2949 raft_consensus.cc:482] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:50.268790 2949 raft_consensus.cc:504] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.269941 2949 leader_election.cc:288] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381), 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915)
I20260619 22:20:50.274628 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.270702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37915 (local address 127.1.74.129:53729)
0619 22:20:50.271074 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.271089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.271212 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.271465 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.271468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.271487 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.271620 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.271627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.272579 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.272582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.273406 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.273414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.274160 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.274164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.274273 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.274300 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.274390 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.274461 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":90,"threads_started":1}
I20260619 22:20:50.274641 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.270834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:53729 (local address 127.1.74.131:37915)
0619 22:20:50.270983 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.270987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.271179 (+ 192us) server_negotiation.cc:407] Connection header received
0619 22:20:50.271300 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.271303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.271347 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.271422 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.271754 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.272466 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.273538 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.274008 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.274134 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.274187 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.274420 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.274509 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:20:50.275130 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.270372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:52061 (local address 127.1.74.130:34381)
0619 22:20:50.270524 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.270528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.270722 (+ 194us) server_negotiation.cc:407] Connection header received
0619 22:20:50.270985 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.270989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.271040 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.271091 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.271554 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.272341 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.273864 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.274412 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.274523 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.274566 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.274923 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.275009 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 22:20:50.275486 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.270300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34381 (local address 127.1.74.129:52061)
0619 22:20:50.270609 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.270620 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.270813 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.271277 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.271280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.271299 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.271421 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.271428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.272475 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.272479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.273736 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.273747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.274634 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.274637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.274738 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.274762 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.275257 (+ 495us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.275326 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260619 22:20:50.275869 2734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99ba9b3cd1949e69079eb06e8500734" candidate_uuid: "6fcb415f4c784660b69dd48d559ae3c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25db844447c94f579ea2087f997f1285" is_pre_election: true
I20260619 22:20:50.275395 2874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99ba9b3cd1949e69079eb06e8500734" candidate_uuid: "6fcb415f4c784660b69dd48d559ae3c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800e2687297c4adcbb2cd149ecf90d29" is_pre_election: true
I20260619 22:20:50.276048 2734 raft_consensus.cc:2454] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fcb415f4c784660b69dd48d559ae3c4 in term 0.
I20260619 22:20:50.276188 2874 raft_consensus.cc:2454] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fcb415f4c784660b69dd48d559ae3c4 in term 0.
I20260619 22:20:50.276408 2525 leader_election.cc:302] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25db844447c94f579ea2087f997f1285, 6fcb415f4c784660b69dd48d559ae3c4; no voters:
I20260619 22:20:50.276729 2949 raft_consensus.cc:2781] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:50.276790 2949 raft_consensus.cc:482] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:50.276835 2949 raft_consensus.cc:3037] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.277838 2949 raft_consensus.cc:504] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.278177 2949 leader_election.cc:288] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [CANDIDATE]: Term 1 election: Requested vote from peers 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381), 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915)
I20260619 22:20:50.278550 2734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99ba9b3cd1949e69079eb06e8500734" candidate_uuid: "6fcb415f4c784660b69dd48d559ae3c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25db844447c94f579ea2087f997f1285"
I20260619 22:20:50.278569 2874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99ba9b3cd1949e69079eb06e8500734" candidate_uuid: "6fcb415f4c784660b69dd48d559ae3c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800e2687297c4adcbb2cd149ecf90d29"
I20260619 22:20:50.278689 2874 raft_consensus.cc:3037] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.278692 2734 raft_consensus.cc:3037] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Advancing to term 1
W20260619 22:20:50.279554 2641 tablet.cc:2369] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:20:50.279783 2734 raft_consensus.cc:2454] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fcb415f4c784660b69dd48d559ae3c4 in term 1.
I20260619 22:20:50.279994 2874 raft_consensus.cc:2454] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fcb415f4c784660b69dd48d559ae3c4 in term 1.
I20260619 22:20:50.280254 2525 leader_election.cc:302] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fcb415f4c784660b69dd48d559ae3c4, 800e2687297c4adcbb2cd149ecf90d29; no voters:
I20260619 22:20:50.280435 2949 raft_consensus.cc:2781] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:50.280658 2949 raft_consensus.cc:686] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 LEADER]: Becoming Leader. State: Replica: 6fcb415f4c784660b69dd48d559ae3c4, State: Running, Role: LEADER
I20260619 22:20:50.280915 2949 consensus_queue.cc:237] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.283595 2435 catalog_manager.cc:5495] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "6fcb415f4c784660b69dd48d559ae3c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } health_report { overall_health: UNKNOWN } } }
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907649, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:50.349872 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.340739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:34571 (local address 127.0.0.1:36418)
0619 22:20:50.341067 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.341083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.341199 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.341482 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.341485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.341960 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:50.342074 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.342081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.342871 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.342873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.343468 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.343475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.343926 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.343929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.344018 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.344576 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:50.344589 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:50.346131 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:50.348056 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:50.348066 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:50.348072 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:50.348333 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:50.348647 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:50.348649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:50.348651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:50.348741 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:50.349287 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:50.349294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:50.349409 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.349521 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.349674 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":131,"threads_started":1}
I20260619 22:20:50.349872 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.340825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.1.74.190:34571)
0619 22:20:50.340983 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.340987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.341177 (+ 190us) server_negotiation.cc:407] Connection header received
0619 22:20:50.341286 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.341288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.341329 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.341411 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:50.342215 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.342753 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.343584 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.343802 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.343885 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.346285 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:50.346288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:50.346290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:50.346312 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:50.347898 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.348451 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.348454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.348455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.348540 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.348903 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.348906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.348907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.349149 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:50.349242 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.349594 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.349740 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":74}
I20260619 22:20:50.354393 2435 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:36418:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 22:20:50.355316 2435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:50.360575 2573 tablet_service.cc:1467] Processing CreateTablet for tablet 1e14b12b71a5451c8e8dddc89aa360ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bebc91c1938e4b318eddbc6ced032c5a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:20:50.360575 2714 tablet_service.cc:1467] Processing CreateTablet for tablet 1e14b12b71a5451c8e8dddc89aa360ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bebc91c1938e4b318eddbc6ced032c5a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:20:50.360791 2854 tablet_service.cc:1467] Processing CreateTablet for tablet 1e14b12b71a5451c8e8dddc89aa360ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bebc91c1938e4b318eddbc6ced032c5a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:20:50.361033 2714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e14b12b71a5451c8e8dddc89aa360ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.361033 2573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e14b12b71a5451c8e8dddc89aa360ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.361212 2854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e14b12b71a5451c8e8dddc89aa360ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:50.363654 2945 tablet_bootstrap.cc:492] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: Bootstrap starting.
I20260619 22:20:50.364034 2944 tablet_bootstrap.cc:492] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: Bootstrap starting.
I20260619 22:20:50.364640 2945 tablet_bootstrap.cc:654] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.364678 2943 tablet_bootstrap.cc:492] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: Bootstrap starting.
I20260619 22:20:50.365000 2944 tablet_bootstrap.cc:654] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.365705 2943 tablet_bootstrap.cc:654] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:50.365885 2945 tablet_bootstrap.cc:492] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: No bootstrap required, opened a new log
I20260619 22:20:50.365962 2945 ts_tablet_manager.cc:1397] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:50.366189 2944 tablet_bootstrap.cc:492] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: No bootstrap required, opened a new log
I20260619 22:20:50.366256 2944 ts_tablet_manager.cc:1397] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:50.366407 2945 raft_consensus.cc:348] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.366528 2945 raft_consensus.cc:374] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.366560 2945 raft_consensus.cc:729] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 800e2687297c4adcbb2cd149ecf90d29, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.366677 2944 raft_consensus.cc:348] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.366705 2945 consensus_queue.cc:260] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.366875 2943 tablet_bootstrap.cc:492] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: No bootstrap required, opened a new log
I20260619 22:20:50.366786 2944 raft_consensus.cc:374] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.366950 2944 raft_consensus.cc:729] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fcb415f4c784660b69dd48d559ae3c4, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.366993 2943 ts_tablet_manager.cc:1397] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:50.367092 2944 consensus_queue.cc:260] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.367221 2961 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.367338 2961 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.367383 2944 ts_tablet_manager.cc:1428] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:50.367456 2943 raft_consensus.cc:348] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.367547 2945 ts_tablet_manager.cc:1428] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:20:50.367568 2943 raft_consensus.cc:374] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:50.367611 2943 raft_consensus.cc:729] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25db844447c94f579ea2087f997f1285, State: Initialized, Role: FOLLOWER
I20260619 22:20:50.367741 2943 consensus_queue.cc:260] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.367937 2949 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.368065 2949 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.368008 2948 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.368144 2948 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.368028 2943 ts_tablet_manager.cc:1428] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:20:50.449983 2948 raft_consensus.cc:482] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:50.450160 2948 raft_consensus.cc:504] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.451301 2948 leader_election.cc:288] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853), 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915)
I20260619 22:20:50.455186 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.451645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:50941 (local address 127.1.74.131:37915)
0619 22:20:50.451789 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.451793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.452054 (+ 261us) server_negotiation.cc:407] Connection header received
0619 22:20:50.452112 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.452136 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.452190 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.452267 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.452541 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.453010 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.454102 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.454527 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.454673 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.454728 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.454940 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.455049 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 22:20:50.455206 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.451612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:58041 (local address 127.1.74.129:37853)
0619 22:20:50.451765 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.451774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.451915 (+ 141us) server_negotiation.cc:407] Connection header received
0619 22:20:50.452016 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.452020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.452067 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.452151 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:50.452499 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.453017 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.454190 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.454560 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.454700 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.454748 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.455014 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.455086 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:20:50.455230 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.451523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37915 (local address 127.1.74.130:50941)
0619 22:20:50.451933 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.451948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.452041 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.452288 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.452291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.452304 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.452413 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.452421 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.453126 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.453132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.453986 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.453995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.454642 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.454648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.454742 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.454773 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.454922 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.455013 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":4,"thread_start_us":70,"threads_started":1}
I20260619 22:20:50.455425 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.451503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:37853 (local address 127.1.74.130:58041)
0619 22:20:50.451807 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.451823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.451924 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.452166 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.452169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.452185 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:50.452327 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.452336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.453124 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.453130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.454061 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.454074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.454707 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.454710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.454878 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.454909 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.454988 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.455052 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":78,"thread_start_us":81,"threads_started":1}
I20260619 22:20:50.455626 2874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e14b12b71a5451c8e8dddc89aa360ec" candidate_uuid: "25db844447c94f579ea2087f997f1285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800e2687297c4adcbb2cd149ecf90d29" is_pre_election: true
I20260619 22:20:50.455830 2874 raft_consensus.cc:2454] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25db844447c94f579ea2087f997f1285 in term 0.
I20260619 22:20:50.455916 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e14b12b71a5451c8e8dddc89aa360ec" candidate_uuid: "25db844447c94f579ea2087f997f1285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcb415f4c784660b69dd48d559ae3c4" is_pre_election: true
I20260619 22:20:50.456050 2594 raft_consensus.cc:2454] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25db844447c94f579ea2087f997f1285 in term 0.
I20260619 22:20:50.456180 2665 leader_election.cc:302] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25db844447c94f579ea2087f997f1285, 800e2687297c4adcbb2cd149ecf90d29; no voters:
I20260619 22:20:50.456431 2948 raft_consensus.cc:2781] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:50.456489 2948 raft_consensus.cc:482] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:50.456548 2948 raft_consensus.cc:3037] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.457494 2948 raft_consensus.cc:504] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.457855 2948 leader_election.cc:288] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [CANDIDATE]: Term 1 election: Requested vote from peers 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853), 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915)
I20260619 22:20:50.458159 2874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e14b12b71a5451c8e8dddc89aa360ec" candidate_uuid: "25db844447c94f579ea2087f997f1285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "800e2687297c4adcbb2cd149ecf90d29"
I20260619 22:20:50.458164 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e14b12b71a5451c8e8dddc89aa360ec" candidate_uuid: "25db844447c94f579ea2087f997f1285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fcb415f4c784660b69dd48d559ae3c4"
I20260619 22:20:50.458274 2594 raft_consensus.cc:3037] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.458274 2874 raft_consensus.cc:3037] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:50.459462 2874 raft_consensus.cc:2454] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25db844447c94f579ea2087f997f1285 in term 1.
I20260619 22:20:50.459462 2594 raft_consensus.cc:2454] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25db844447c94f579ea2087f997f1285 in term 1.
I20260619 22:20:50.459726 2666 leader_election.cc:302] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25db844447c94f579ea2087f997f1285, 6fcb415f4c784660b69dd48d559ae3c4; no voters:
I20260619 22:20:50.459926 2948 raft_consensus.cc:2781] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:50.460145 2948 raft_consensus.cc:686] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [term 1 LEADER]: Becoming Leader. State: Replica: 25db844447c94f579ea2087f997f1285, State: Running, Role: LEADER
I20260619 22:20:50.460397 2948 consensus_queue.cc:237] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } }
I20260619 22:20:50.461633 2987 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: TxnStatusTablet state changed. Reason: New leader 25db844447c94f579ea2087f997f1285. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.461815 2987 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:20:50.462534 2989 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 22:20:50.462682 2989 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 22:20:50.462916 2436 catalog_manager.cc:5495] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25db844447c94f579ea2087f997f1285 (127.1.74.130). New cstate: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } health_report { overall_health: UNKNOWN } } }
I20260619 22:20:50.495005 2981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.491277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34381 (local address 127.0.0.1:56718)
0619 22:20:50.491429 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.491447 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.491551 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.491880 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.491883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.491893 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:50.492003 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.492014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.492731 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.492736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.493383 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.493392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.493856 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.493860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.493949 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.494000 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.494698 (+ 698us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.494714 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.494782 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.494893 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 22:20:50.495005 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.491451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56718 (local address 127.1.74.130:34381)
0619 22:20:50.491577 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.491582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.491599 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:20:50.491693 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.491696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.491759 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.491867 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:50.492115 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.492620 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.493505 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.493747 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.493856 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.494163 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.494580 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.494700 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.494791 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.494862 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:20:50.497740 2594 raft_consensus.cc:1261] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Refusing update from remote peer 25db844447c94f579ea2087f997f1285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:50.497965 2874 raft_consensus.cc:1261] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Refusing update from remote peer 25db844447c94f579ea2087f997f1285: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:50.498265 2948 consensus_queue.cc:1035] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:50.498449 2987 consensus_queue.cc:1035] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:50.499802 2949 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: TxnStatusTablet state changed. Reason: New leader 25db844447c94f579ea2087f997f1285. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.499938 2949 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.500051 2961 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: TxnStatusTablet state changed. Reason: New leader 25db844447c94f579ea2087f997f1285. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.500216 2961 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.502902 2987 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.503031 2987 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:20:50.503573 2961 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.503682 2961 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 800e2687297c4adcbb2cd149ecf90d29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.504642 2949 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.504757 2949 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 6fcb415f4c784660b69dd48d559ae3c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:50.502911 2948 tablet_replica.cc:439] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "25db844447c94f579ea2087f997f1285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fcb415f4c784660b69dd48d559ae3c4" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 37853 } } peers { permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 } } peers { permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 } } }
I20260619 22:20:50.505286 2948 tablet_replica.cc:442] T 1e14b12b71a5451c8e8dddc89aa360ec P 25db844447c94f579ea2087f997f1285: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:20:50.513500 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.509516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.1.74.129:37853)
0619 22:20:50.509823 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.509826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.509836 (+ 10us) server_negotiation.cc:407] Connection header received
0619 22:20:50.509899 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.509901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.509960 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.510077 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:50.510324 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.510989 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.511845 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.512195 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.512315 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.512605 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.513063 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.513195 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.513291 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.513353 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260619 22:20:50.524946 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.520773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:49821 (local address 127.1.74.130:34381)
0619 22:20:50.520925 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.520929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.521050 (+ 121us) server_negotiation.cc:407] Connection header received
0619 22:20:50.521146 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.521148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.521189 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.521249 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:50.521515 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.522002 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.523766 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.524128 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.524225 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.524544 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.524615 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.524676 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.524789 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.524836 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 22:20:50.525446 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.520620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34381 (local address 127.1.74.129:49821)
0619 22:20:50.520940 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.520955 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.521062 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.521279 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.521281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.521289 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:50.521419 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.521425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.522121 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.522124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.523603 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.523613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.524245 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.524249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.524347 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.524403 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.524699 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.524705 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.524770 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.524847 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
I20260619 22:20:50.533571 2874 raft_consensus.cc:1261] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Refusing update from remote peer 6fcb415f4c784660b69dd48d559ae3c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:50.533600 2734 raft_consensus.cc:1261] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Refusing update from remote peer 6fcb415f4c784660b69dd48d559ae3c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:50.534051 2974 consensus_queue.cc:1035] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "800e2687297c4adcbb2cd149ecf90d29" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:50.534246 2949 consensus_queue.cc:1035] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25db844447c94f579ea2087f997f1285" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:50.539633 2956 mvcc.cc:205] Tried to move back new op lower bound from 7298693736581455872 to 7298693735553318912. Current Snapshot: MvccSnapshot[applied={T|T < 7298693736581455872}]
I20260619 22:20:50.559934 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.556445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:57489 (local address 127.1.74.129:37853)
0619 22:20:50.556562 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.556566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.556582 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:20:50.556888 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.556892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.556940 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.557036 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:50.557273 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.557782 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.558638 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.558973 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.559093 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.559417 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.559512 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.559584 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.559748 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.559809 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 22:20:50.559963 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.556102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:37853 (local address 127.1.74.130:57489)
0619 22:20:50.556387 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:50.556403 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:50.556768 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:50.557051 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:50.557054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:50.557062 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:50.557168 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.557173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.557898 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.557902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.558519 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:50.558529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.559094 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.559098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:50.559181 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.559248 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:50.559624 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:50.559630 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 22:20:50.559718 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 22:20:50.559800 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260619 22:20:50.593974 2436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:50.594187 2436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:50.596926 2436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6ed84a1f38f48e68738b861f8ad08ce: Sending DeleteTablet for 3 replicas of tablet c99ba9b3cd1949e69079eb06e8500734
I20260619 22:20:50.597832 2714 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ba9b3cd1949e69079eb06e8500734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:50 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56702
I20260619 22:20:50.597997 2573 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ba9b3cd1949e69079eb06e8500734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:50 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56000
I20260619 22:20:50.598222 2854 tablet_service.cc:1514] Processing DeleteTablet for tablet c99ba9b3cd1949e69079eb06e8500734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:50 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58170
I20260619 22:20:50.598438 2967 tablet_replica.cc:330] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: stopping tablet replica
I20260619 22:20:50.598579 2967 raft_consensus.cc:2229] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:50.598730 2967 raft_consensus.cc:2258] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:50.598941 2968 tablet_replica.cc:330] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: stopping tablet replica
I20260619 22:20:50.599061 2968 raft_consensus.cc:2229] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:20:50.599383 2968 raft_consensus.cc:2258] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:50.599588 2966 tablet_replica.cc:330] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: stopping tablet replica
I20260619 22:20:50.599725 2966 raft_consensus.cc:2229] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:50.599851 2966 raft_consensus.cc:2258] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:50.601091 2966 ts_tablet_manager.cc:1905] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:50.603643 2967 ts_tablet_manager.cc:1905] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:50.606740 2967 ts_tablet_manager.cc:1918] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:50.606859 2967 log.cc:1196] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/c99ba9b3cd1949e69079eb06e8500734
I20260619 22:20:50.607198 2967 ts_tablet_manager.cc:1939] T c99ba9b3cd1949e69079eb06e8500734 P 25db844447c94f579ea2087f997f1285: Deleting consensus metadata
I20260619 22:20:50.608047 2421 catalog_manager.cc:4841] TS 25db844447c94f579ea2087f997f1285 (127.1.74.130:34381): tablet c99ba9b3cd1949e69079eb06e8500734 (table test-table [id=8ee81a2d5b884a88ba93f83f3162c372]) successfully deleted
I20260619 22:20:50.608800 2968 ts_tablet_manager.cc:1905] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:50.611829 2968 ts_tablet_manager.cc:1918] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:50.611923 2968 log.cc:1196] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/c99ba9b3cd1949e69079eb06e8500734
I20260619 22:20:50.612234 2968 ts_tablet_manager.cc:1939] T c99ba9b3cd1949e69079eb06e8500734 P 6fcb415f4c784660b69dd48d559ae3c4: Deleting consensus metadata
I20260619 22:20:50.613119 2422 catalog_manager.cc:4841] TS 6fcb415f4c784660b69dd48d559ae3c4 (127.1.74.129:37853): tablet c99ba9b3cd1949e69079eb06e8500734 (table test-table [id=8ee81a2d5b884a88ba93f83f3162c372]) successfully deleted
I20260619 22:20:50.615831 2966 ts_tablet_manager.cc:1918] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:50.615928 2966 log.cc:1196] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/c99ba9b3cd1949e69079eb06e8500734
I20260619 22:20:50.616236 2966 ts_tablet_manager.cc:1939] T c99ba9b3cd1949e69079eb06e8500734 P 800e2687297c4adcbb2cd149ecf90d29: Deleting consensus metadata
I20260619 22:20:50.616991 2421 catalog_manager.cc:4841] TS 800e2687297c4adcbb2cd149ecf90d29 (127.1.74.131:37915): tablet c99ba9b3cd1949e69079eb06e8500734 (table test-table [id=8ee81a2d5b884a88ba93f83f3162c372]) successfully deleted
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[2392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:20:50.631208 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.614506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.1.74.129:37853)
0619 22:20:50.617286 (+ 2780us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.617292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.617710 (+ 418us) server_negotiation.cc:407] Connection header received
0619 22:20:50.617924 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.617927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.617978 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.618032 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:50.619185 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.620046 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.621403 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.621803 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.621885 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.626349 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:50.626363 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:50.626382 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:50.626419 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:50.628653 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.629224 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.629232 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.629234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.629344 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.629767 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.629771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.629773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.630117 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:50.630354 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.630739 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.630983 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2682}
W20260619 22:20:50.632972 2634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56028
I20260619 22:20:50.645987 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:50.634911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.1.74.190:34571)
0619 22:20:50.635038 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:50.635044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:50.635513 (+ 469us) server_negotiation.cc:407] Connection header received
0619 22:20:50.635633 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:50.635636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:50.635691 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:50.635774 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:50.636640 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.637443 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.638683 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:50.639087 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:50.639163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:50.641433 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:50.641437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:50.641440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:50.641474 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:50.643929 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.644485 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.644489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.644491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.644580 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:50.644908 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:50.644912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:50.644914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:50.645150 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:50.645271 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:50.645659 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 22:20:50.645793 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 22:20:50.646909 2436 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36426
I20260619 22:20:50.648520 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 2509
I20260619 22:20:50.657361 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 2649
I20260619 22:20:50.665033 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 2789
I20260619 22:20:50.671829 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 2405
2026-06-19T22:20:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2801 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[3011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[3011](info): set up 2 sockets
Jun 19 22:20:50 dist-test-slave-k84h krb5kdc[3011](info): commencing operation
krb5kdc: starting...
W20260619 22:20:51.726678 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:20:51 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:20:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:20:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:20:51.854001 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40475
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:51.968652 3024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:51.968958 3024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:51.969000 3024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:51.973142 3024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:20:51.973217 3024 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 22:20:51.973237 3024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:51.973256 3024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:51.973273 3024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:20:51.973289 3024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:20:51.978741 3024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40475
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:51.979827 3024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:51.980870 3024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:51.989341 3029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:51.989351 3030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:51.989363 3032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:51.989774 3024 server_base.cc:1034] running on GCE node
I20260619 22:20:51.990314 3024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:51.991497 3024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:51.992686 3024 hybrid_clock.cc:648] HybridClock initialized: now 1781907651992673 us; error 29 us; skew 500 ppm
Jun 19 22:20:51 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907651, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:51.995785 3024 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:20:51.996680 3024 webserver.cc:460] Webserver started at http://127.1.74.190:46117/ using document root <none> and password file <none>
I20260619 22:20:51.997215 3024 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:51.997275 3024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:51.997440 3024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:51.999514 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "e2b018f052bd4b95b6d052b28d3edfb6"
format_stamp: "Formatted at 2026-06-19 22:20:51 on dist-test-slave-k84h"
server_key: "db1eb647d9ada0566a752f7abcd82183"
server_key_iv: "867f93caad2a1a532122b3a6cec496bb"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.000032 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "e2b018f052bd4b95b6d052b28d3edfb6"
format_stamp: "Formatted at 2026-06-19 22:20:51 on dist-test-slave-k84h"
server_key: "db1eb647d9ada0566a752f7abcd82183"
server_key_iv: "867f93caad2a1a532122b3a6cec496bb"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.003978 3024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:20:52.006560 3039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.007929 3024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:20:52.008078 3024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "e2b018f052bd4b95b6d052b28d3edfb6"
format_stamp: "Formatted at 2026-06-19 22:20:51 on dist-test-slave-k84h"
server_key: "db1eb647d9ada0566a752f7abcd82183"
server_key_iv: "867f93caad2a1a532122b3a6cec496bb"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.008209 3024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:52.035706 3024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:52.038548 3024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:52.038755 3024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:52.046533 3024 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40475
I20260619 22:20:52.046561 3101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40475 every 8 connection(s)
I20260619 22:20:52.047470 3024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:20:52.050163 3102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:52.051051 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3024
I20260619 22:20:52.051142 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:20:52.051438 1322 external_mini_cluster.cc:1428] Setting key f1349c6df3878a7c405f055096f20ba9
I20260619 22:20:52.057308 3102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Bootstrap starting.
I20260619 22:20:52.060076 3102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Neither blocks nor log segments found. Creating new log.
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:52.060854 3102 log.cc:824] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:52.063591 3102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: No bootstrap required, opened a new log
I20260619 22:20:52.066962 3102 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40475 } }
I20260619 22:20:52.067166 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.052835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.1.74.190:40475)
0619 22:20:52.053268 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.053289 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.053337 (+ 48us) server_negotiation.cc:407] Connection header received
0619 22:20:52.054194 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.054217 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.054643 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.055060 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.055701 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.057208 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.058120 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.058822 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.059054 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.061550 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.061556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.061569 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.061599 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.064318 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.064762 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.064768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.064781 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.064889 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.065175 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.065179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.065180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.065579 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.065815 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.066074 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.066355 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
I20260619 22:20:52.067241 3102 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:52.067312 3102 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b018f052bd4b95b6d052b28d3edfb6, State: Initialized, Role: FOLLOWER
I20260619 22:20:52.067837 3102 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40475 } }
I20260619 22:20:52.067994 3102 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:20:52.068051 3102 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:20:52.068141 3102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:52.069164 3102 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40475 } }
I20260619 22:20:52.069587 3102 leader_election.cc:302] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2b018f052bd4b95b6d052b28d3edfb6; no voters:
I20260619 22:20:52.069965 3102 leader_election.cc:288] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:20:52.070134 3107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:52.070399 3107 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [term 1 LEADER]: Becoming Leader. State: Replica: e2b018f052bd4b95b6d052b28d3edfb6, State: Running, Role: LEADER
I20260619 22:20:52.070703 3107 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40475 } }
I20260619 22:20:52.071365 3102 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:20:52.072516 3108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40475 } } }
I20260619 22:20:52.072645 3108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:52.072649 3109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2b018f052bd4b95b6d052b28d3edfb6. Latest consensus state: current_term: 1 leader_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b018f052bd4b95b6d052b28d3edfb6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40475 } } }
I20260619 22:20:52.072784 3109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6 [sys.catalog]: This master's current role is: LEADER
I20260619 22:20:52.073216 3116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:20:52.076521 3116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:20:52.082098 3116 catalog_manager.cc:1347] Generated new cluster ID: ccf409c436b44b5eb8f49f1c52bcedb6
I20260619 22:20:52.082177 3116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:20:52.087184 3116 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:20:52.088193 3116 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:20:52.096330 3116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Generated new TSK 0
I20260619 22:20:52.097078 3116 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:20:52.130401 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40475
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:52.244513 3128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:52.244798 3128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:52.244848 3128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:52.249063 3128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:52.249133 3128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:52.249176 3128 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 22:20:52.249189 3128 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:20:52.249222 3128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:20:52.254638 3128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:52.255790 3128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:52.256832 3128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:52.266674 3136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:52.266641 3134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:52.266641 3133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:52.266903 3128 server_base.cc:1034] running on GCE node
I20260619 22:20:52.267426 3128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:52.268249 3128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:52.269455 3128 hybrid_clock.cc:648] HybridClock initialized: now 1781907652269427 us; error 52 us; skew 500 ppm
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:52.272708 3128 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:20:52.273674 3128 webserver.cc:460] Webserver started at http://127.1.74.129:40217/ using document root <none> and password file <none>
I20260619 22:20:52.274277 3128 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:52.274339 3128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:52.274559 3128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:52.276682 3128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "505fb9b20898215210d0b7bf017e1f2e"
server_key_iv: "0d13e0cd9a54e24dffc6918d4fbc99ca"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.277288 3128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "505fb9b20898215210d0b7bf017e1f2e"
server_key_iv: "0d13e0cd9a54e24dffc6918d4fbc99ca"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.281179 3128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:20:52.283982 3143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.285329 3128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:20:52.285480 3128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "505fb9b20898215210d0b7bf017e1f2e"
server_key_iv: "0d13e0cd9a54e24dffc6918d4fbc99ca"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.285609 3128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:52.298802 3128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:52.301975 3128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:52.302198 3128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:52.303758 3128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:52.303838 3128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.303915 3128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:52.303960 3128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:52.315413 3128 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45527
I20260619 22:20:52.315481 3258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45527 every 8 connection(s)
I20260619 22:20:52.316490 3128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:20:52.317741 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3128
I20260619 22:20:52.317889 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:20:52.318156 1322 external_mini_cluster.cc:1428] Setting key 7a75939822b20b783afa9d952b543504
I20260619 22:20:52.318943 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.304687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:45691 (local address 127.1.74.190:40475)
0619 22:20:52.304815 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.304819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.305908 (+ 1089us) server_negotiation.cc:407] Connection header received
0619 22:20:52.307173 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.307179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.307253 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.307362 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.308914 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.309830 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.311243 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.311624 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.311751 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.314741 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.314745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.314747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.314772 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.316476 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.317154 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.317159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.317163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.317264 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.317667 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.317670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.317671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.317888 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.317986 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.318615 (+ 629us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.318765 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:20:52.319859 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.305269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.1.74.129:45691)
0619 22:20:52.305752 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.305805 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.306795 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.307611 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.307621 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.308109 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:52.308762 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.308786 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.309968 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.309974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.311111 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.311120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.311804 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.311809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.311995 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.312636 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:52.312652 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:52.314560 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:52.316635 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.316640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.316655 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.317025 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.317367 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.317370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.317372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.317577 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.318047 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:52.318052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:52.318331 (+ 279us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.318495 (+ 164us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.318831 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":149,"threads_started":1}
I20260619 22:20:52.321365 3259 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40475
I20260619 22:20:52.321561 3259 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:52.322139 3259 heartbeater.cc:511] Master 127.1.74.190:40475 requested a full tablet report, sending...
I20260619 22:20:52.323742 3056 ts_manager.cc:194] Registered new tserver with Master: a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527)
I20260619 22:20:52.325160 3056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:45691
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:20:52.332432 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.324085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.1.74.129:39083)
0619 22:20:52.324382 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.324399 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.324505 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.324752 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.324757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.325015 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:52.325171 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.325182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.326239 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.326246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.326900 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.326914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.327483 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.327488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.327626 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.328204 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:52.328224 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:52.328704 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:52.330659 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.330665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.330670 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.330960 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.331320 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.331324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.331329 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.331416 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.331846 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:52.331848 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:52.331934 (+ 86us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.332065 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.332191 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":99,"threads_started":1}
I20260619 22:20:52.332448 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.323963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:39083 (local address 127.1.74.190:40475)
0619 22:20:52.324102 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.324108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.324486 (+ 378us) server_negotiation.cc:407] Connection header received
0619 22:20:52.324586 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.324589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.324634 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.324718 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.325379 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.326095 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.327038 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.327344 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.327452 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.328865 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.328869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.328872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.328900 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.330520 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.331095 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.331099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.331101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.331189 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.331519 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.331523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.331524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.331727 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.331815 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.332126 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.332232 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:20:52.350572 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40475
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:52.461771 3268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:52.462056 3268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:52.462105 3268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:52.466184 3268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:52.466240 3268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:52.466281 3268 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 22:20:52.466295 3268 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:20:52.466327 3268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:20:52.471655 3268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:52.472807 3268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:52.473861 3268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:52.483731 3274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:52.483783 3276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:52.483731 3273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:52.483939 3268 server_base.cc:1034] running on GCE node
I20260619 22:20:52.484440 3268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:52.485101 3268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:52.486274 3268 hybrid_clock.cc:648] HybridClock initialized: now 1781907652486257 us; error 32 us; skew 500 ppm
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:52.489204 3268 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:20:52.490226 3268 webserver.cc:460] Webserver started at http://127.1.74.130:37871/ using document root <none> and password file <none>
I20260619 22:20:52.490773 3268 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:52.490831 3268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:52.491079 3268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:52.493156 3268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "62a19c3e7c174cd6bfebba1aade3234e"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "4bebc13f8a9b26037f241730ecebfcfd"
server_key_iv: "a71164fe8649cc2b8429871a83e05b2b"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.493692 3268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "62a19c3e7c174cd6bfebba1aade3234e"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "4bebc13f8a9b26037f241730ecebfcfd"
server_key_iv: "a71164fe8649cc2b8429871a83e05b2b"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.497717 3268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260619 22:20:52.500308 3283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.501596 3268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:20:52.501703 3268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "62a19c3e7c174cd6bfebba1aade3234e"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "4bebc13f8a9b26037f241730ecebfcfd"
server_key_iv: "a71164fe8649cc2b8429871a83e05b2b"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.501809 3268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:52.512598 3268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:52.515661 3268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:52.515820 3268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:52.517303 3268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:52.517376 3268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.517429 3268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:52.517459 3268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:52.528378 3268 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:45893
I20260619 22:20:52.528494 3398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:45893 every 8 connection(s)
I20260619 22:20:52.529294 3268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:20:52.532586 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.517906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:51949 (local address 127.1.74.190:40475)
0619 22:20:52.518036 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.518040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.519764 (+ 1724us) server_negotiation.cc:407] Connection header received
0619 22:20:52.520960 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.520963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.521029 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.521172 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.522950 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.523561 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.525373 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.525593 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.525675 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.528889 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.528893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.528896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.528930 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.530587 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.531225 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.531228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.531230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.531325 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.531647 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.531650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.531651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.531827 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.531901 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.532357 (+ 456us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.532466 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:20:52.533432 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.519046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.1.74.130:51949)
0619 22:20:52.519614 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.519658 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.520731 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.521364 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.521375 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.521966 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:52.522748 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.522772 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.523698 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.523702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.525246 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.525256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.525753 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.525758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.525970 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.526773 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:52.526818 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:52.528717 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:52.530714 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.530722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.530739 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.531071 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.531414 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.531421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.531422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.531550 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.531918 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:52.531923 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:52.532150 (+ 227us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.532296 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.532547 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":143,"threads_started":1}
I20260619 22:20:52.534564 3399 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40475
I20260619 22:20:52.534724 3399 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:52.535408 3399 heartbeater.cc:511] Master 127.1.74.190:40475 requested a full tablet report, sending...
I20260619 22:20:52.536576 3055 ts_manager.cc:194] Registered new tserver with Master: 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893)
I20260619 22:20:52.537279 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3268
I20260619 22:20:52.537384 3055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:51949
I20260619 22:20:52.537438 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:20:52.537806 1322 external_mini_cluster.cc:1428] Setting key 61c1eb15a0b10c29550e3d1ac6c1d6d7
I20260619 22:20:52.544929 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.537167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:58293 (local address 127.1.74.190:40475)
0619 22:20:52.537297 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.537304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.537487 (+ 183us) server_negotiation.cc:407] Connection header received
0619 22:20:52.537649 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.537653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.537709 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.537808 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.538528 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.539285 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.540127 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.540521 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.540657 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.541788 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.541791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.541794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.541817 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.543309 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.543725 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.543729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.543731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.543808 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.544087 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.544092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.544094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.544298 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.544406 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.544663 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.544771 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:20:52.544934 3405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.537047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.1.74.130:58293)
0619 22:20:52.537405 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.537423 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.537550 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.537999 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.538002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.538238 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:52.538390 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.538400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.539437 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.539440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.540005 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.540012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.540638 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.540641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.540706 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.541246 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:52.541260 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:52.541645 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:52.543423 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.543428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.543430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.543627 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.543906 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.543909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.543911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.543998 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.544396 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:52.544399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:52.544496 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.544605 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.544753 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:20:52.567318 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40475
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:20:52.691216 3408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:20:52.691527 3408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:20:52.691622 3408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:20:52.695894 3408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:20:52.695973 3408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:20:52.696048 3408 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 22:20:52.696079 3408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:20:52.696167 3408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:20:52.701820 3408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:40475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:20:52.703083 3408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:20:52.704171 3408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:20:52.714665 3413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:52.714665 3416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:20:52.714763 3414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:20:52.714726 3408 server_base.cc:1034] running on GCE node
I20260619 22:20:52.715308 3408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:20:52.716073 3408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:20:52.717264 3408 hybrid_clock.cc:648] HybridClock initialized: now 1781907652717231 us; error 51 us; skew 500 ppm
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:20:52.720322 3408 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:20:52.721366 3408 webserver.cc:460] Webserver started at http://127.1.74.131:45005/ using document root <none> and password file <none>
I20260619 22:20:52.722002 3408 fs_manager.cc:359] Metadata directory not provided
I20260619 22:20:52.722101 3408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:20:52.722342 3408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:20:52.724581 3408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "8635171b39d042dfa798b94e72dc7f4d"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "cf382059f76d825ffa333a34b1033f72"
server_key_iv: "21fbf9117d69494c0b4c50b67a366980"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.725186 3408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "8635171b39d042dfa798b94e72dc7f4d"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "cf382059f76d825ffa333a34b1033f72"
server_key_iv: "21fbf9117d69494c0b4c50b67a366980"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.729104 3408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:20:52.731696 3423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.732968 3408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:20:52.733112 3408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "8635171b39d042dfa798b94e72dc7f4d"
format_stamp: "Formatted at 2026-06-19 22:20:52 on dist-test-slave-k84h"
server_key: "cf382059f76d825ffa333a34b1033f72"
server_key_iv: "21fbf9117d69494c0b4c50b67a366980"
server_key_version: "encryptionkey@0"
I20260619 22:20:52.733237 3408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:20:52.759366 3408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:20:52.762887 3408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:20:52.763110 3408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:20:52.764689 3408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:20:52.764772 3408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:20:52.764840 3408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:20:52.764881 3408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:52.776011 3408 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:35127
I20260619 22:20:52.776094 3538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:35127 every 8 connection(s)
I20260619 22:20:52.777109 3408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:20:52.779398 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.765681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:47573 (local address 127.1.74.190:40475)
0619 22:20:52.765813 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.765817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.767006 (+ 1189us) server_negotiation.cc:407] Connection header received
0619 22:20:52.768042 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.768046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.768139 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.768227 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.769834 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.770744 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.772539 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.772900 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.773008 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.775676 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.775679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.775682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.775716 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.777371 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.777967 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.777970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.777971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.778042 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.778346 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.778348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.778350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.778524 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.778600 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.779160 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.779264 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:20:52.780160 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.766229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.1.74.131:47573)
0619 22:20:52.766795 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.766889 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.767825 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.768456 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.768465 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.769010 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:52.769679 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.769700 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.770913 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.770919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.772405 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.772418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.773041 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.773044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.773182 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.773811 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:52.773827 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:52.775505 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:52.777502 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.777507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.777523 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.777837 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.778126 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.778129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.778130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.778253 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.778613 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:52.778618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:52.778836 (+ 218us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.779056 (+ 220us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.779320 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":153,"threads_started":1}
I20260619 22:20:52.781432 3539 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40475
I20260619 22:20:52.781594 3539 heartbeater.cc:464] Registering TS with master...
I20260619 22:20:52.782079 3539 heartbeater.cc:511] Master 127.1.74.190:40475 requested a full tablet report, sending...
I20260619 22:20:52.783289 3055 ts_manager.cc:194] Registered new tserver with Master: 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:52.784020 3055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:47573
I20260619 22:20:52.784715 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3408
I20260619 22:20:52.784834 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:20:52.785116 1322 external_mini_cluster.cc:1428] Setting key e5120a73dd47a875d019101e9b291558
I20260619 22:20:52.787182 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:20:52.791594 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.783771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.1.74.131:43703)
0619 22:20:52.784058 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.784074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.784173 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.784404 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.784406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.784671 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:52.784811 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.784820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.785802 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.785805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.786591 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.786600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.787167 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.787170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.787272 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.787751 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:52.787767 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:52.788226 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:52.789994 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.789997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.789999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.790210 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.790520 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:52.790523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:52.790525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:52.790606 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:52.791037 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:52.791040 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:52.791142 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.791253 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.791408 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":112,"threads_started":1}
I20260619 22:20:52.791823 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.783828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:43703 (local address 127.1.74.190:40475)
0619 22:20:52.783956 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.783960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.784171 (+ 211us) server_negotiation.cc:407] Connection header received
0619 22:20:52.784259 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.784261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.784320 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.784396 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.784944 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.785660 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.786698 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.787029 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.787134 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.788386 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.788389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.788392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.788417 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.789875 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.790307 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.790312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.790314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.790397 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.790696 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.790700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.790703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.790939 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.791048 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.791303 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.791414 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 22:20:52 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:52.813629 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.805150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.1.74.190:40475)
0619 22:20:52.805288 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.805292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.805306 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:20:52.805350 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.805352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.805391 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.805461 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:52.806199 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.806730 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.807637 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.807862 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.807947 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.810417 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:52.810420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:52.810422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:52.810447 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:52.811953 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.812382 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.812385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.812386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.812459 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:52.812757 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:52.812759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:52.812761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:52.812949 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:52.813051 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.813351 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.813449 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:20:52.816545 3055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:20:52.819595 3055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:52.834182 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.828526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53872 (local address 127.1.74.129:45527)
0619 22:20:52.828935 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.828942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.828966 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:20:52.829095 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.829102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.829337 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.829499 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:52.829995 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.831219 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.832706 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.833477 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.833587 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.833743 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.833928 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.834029 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":97,"threads_started":1}
I20260619 22:20:52.834232 3556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.828364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.0.0.1:53872)
0619 22:20:52.828758 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.828814 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.828949 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.829681 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.829690 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.829719 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:52.829852 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.829859 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.831343 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.831347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.832568 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.832578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.833596 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.833600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.833719 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.833763 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.833885 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.834035 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260619 22:20:52.834934 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.829202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.1.74.130:45893)
0619 22:20:52.829543 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.829552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.829956 (+ 404us) server_negotiation.cc:407] Connection header received
0619 22:20:52.830040 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.830047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.830226 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.830425 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:52.830716 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.831675 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.833103 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.833949 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.834440 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.834606 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.834668 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.834756 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":101,"threads_started":1}
I20260619 22:20:52.835002 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.829120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:35127 (local address 127.0.0.1:58060)
0619 22:20:52.829955 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.829968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.830048 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.830742 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.830744 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.830758 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:52.830820 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.830824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.832095 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.832098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.833372 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.833380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.834502 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.834506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.834604 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.834654 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.834726 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.834804 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":112,"threads_started":1}
I20260619 22:20:52.835167 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.829307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.1.74.131:35127)
0619 22:20:52.829881 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:52.829895 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:52.830134 (+ 239us) server_negotiation.cc:407] Connection header received
0619 22:20:52.830226 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:52.830231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:52.830431 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:52.830590 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:52.830953 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.831979 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.833501 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.834288 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.834400 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.834565 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:52.834911 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 22:20:52.835003 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":105,"threads_started":1}
I20260619 22:20:52.834797 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:52.828727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.0.0.1:41644)
0619 22:20:52.829648 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:52.829658 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:52.829777 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:52.830428 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:52.830432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:52.830450 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:52.830583 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.830589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.831795 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.831800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.832964 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:52.832975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:52.834078 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:52.834084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:52.834180 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:52.834201 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:20:52.834253 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 22:20:52.834326 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":98,"threads_started":1}
I20260619 22:20:52.838371 3333 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 (DEFAULT_TABLE table=test-table [id=adbfa792175b40a9a138a2acd8439a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:52.838343 3193 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 (DEFAULT_TABLE table=test-table [id=adbfa792175b40a9a138a2acd8439a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:52.838477 3473 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 (DEFAULT_TABLE table=test-table [id=adbfa792175b40a9a138a2acd8439a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:52.839373 3333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:52.839395 3193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:52.839402 3473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:52.846510 3562 tablet_bootstrap.cc:492] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Bootstrap starting.
I20260619 22:20:52.847151 3563 tablet_bootstrap.cc:492] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Bootstrap starting.
I20260619 22:20:52.847890 3564 tablet_bootstrap.cc:492] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Bootstrap starting.
I20260619 22:20:52.848796 3562 tablet_bootstrap.cc:654] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:52.849205 3563 tablet_bootstrap.cc:654] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:52.849577 3562 log.cc:824] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:52.849789 3564 tablet_bootstrap.cc:654] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:52.849997 3563 log.cc:824] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:52.850566 3564 log.cc:824] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Log is configured to *not* fsync() on all Append() calls
I20260619 22:20:52.851850 3563 tablet_bootstrap.cc:492] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: No bootstrap required, opened a new log
I20260619 22:20:52.851950 3562 tablet_bootstrap.cc:492] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: No bootstrap required, opened a new log
I20260619 22:20:52.852085 3563 ts_tablet_manager.cc:1397] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 22:20:52.852128 3562 ts_tablet_manager.cc:1397] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:20:52.852344 3564 tablet_bootstrap.cc:492] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: No bootstrap required, opened a new log
I20260619 22:20:52.852501 3564 ts_tablet_manager.cc:1397] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 22:20:52.855319 3563 raft_consensus.cc:348] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:52.855571 3563 raft_consensus.cc:374] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:52.855630 3563 raft_consensus.cc:729] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c083b15f3b485b9cf921aa1cb042b2, State: Initialized, Role: FOLLOWER
I20260619 22:20:52.856061 3563 consensus_queue.cc:260] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:52.856096 3564 raft_consensus.cc:348] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:52.856083 3562 raft_consensus.cc:348] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:52.856329 3562 raft_consensus.cc:374] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:52.856329 3564 raft_consensus.cc:374] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:52.856391 3562 raft_consensus.cc:729] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Initialized, Role: FOLLOWER
I20260619 22:20:52.856391 3564 raft_consensus.cc:729] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8635171b39d042dfa798b94e72dc7f4d, State: Initialized, Role: FOLLOWER
I20260619 22:20:52.856796 3562 consensus_queue.cc:260] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:52.856796 3564 consensus_queue.cc:260] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:52.856858 3259 heartbeater.cc:503] Master 127.1.74.190:40475 was elected leader, sending a full tablet report...
I20260619 22:20:52.857053 3563 ts_tablet_manager.cc:1428] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 22:20:52.857769 3539 heartbeater.cc:503] Master 127.1.74.190:40475 was elected leader, sending a full tablet report...
I20260619 22:20:52.858268 3564 ts_tablet_manager.cc:1428] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:20:52.858713 3399 heartbeater.cc:503] Master 127.1.74.190:40475 was elected leader, sending a full tablet report...
I20260619 22:20:52.859349 3562 ts_tablet_manager.cc:1428] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
W20260619 22:20:53.028348 3540 tablet.cc:2369] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:20:53.030759 3400 tablet.cc:2369] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:20:53.067884 3260 tablet.cc:2369] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:20:53.118805 3569 raft_consensus.cc:482] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:53.119163 3569 raft_consensus.cc:504] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:53.120719 3569 leader_election.cc:288] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:53.124769 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.121499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:46975 (local address 127.1.74.131:35127)
0619 22:20:53.121646 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.121651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.121665 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:20:53.121804 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.121810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.121861 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.121928 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:53.122183 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.122694 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.123812 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.124210 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.124341 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.124396 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.124559 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.124639 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:20:53.124987 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.121100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:55137 (local address 127.1.74.129:45527)
0619 22:20:53.121273 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.121276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.121317 (+ 41us) server_negotiation.cc:407] Connection header received
0619 22:20:53.121455 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.121459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.121515 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.121615 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:53.121952 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.122709 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.123893 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.124369 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.124516 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.124573 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.124779 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.124852 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 22:20:53.124812 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.121346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:35127 (local address 127.1.74.130:46975)
0619 22:20:53.121576 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.121594 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.121671 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.121947 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.121949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.121965 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:53.122079 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.122084 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.122803 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.122810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.123661 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.123681 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.124328 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.124331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.124430 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.124451 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.124546 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.124667 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":43,"thread_start_us":61,"threads_started":1}
I20260619 22:20:53.125252 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.120950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.1.74.130:55137)
0619 22:20:53.121211 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.121226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.121334 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.121627 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.121631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.121650 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:53.121819 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.121826 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.122828 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.122833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.123753 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.123763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.124503 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.124507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.124642 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.124671 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.124763 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.124838 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"spinlock_wait_cycles":2944,"thread_start_us":96,"threads_started":1}
I20260619 22:20:53.125845 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf0ca38d77f4faa9bfb9fc984d856e0" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d" is_pre_election: true
I20260619 22:20:53.125883 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf0ca38d77f4faa9bfb9fc984d856e0" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" is_pre_election: true
I20260619 22:20:53.126145 3493 raft_consensus.cc:2454] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 0.
I20260619 22:20:53.126190 3213 raft_consensus.cc:2454] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 0.
I20260619 22:20:53.126628 3286 leader_election.cc:302] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, 8635171b39d042dfa798b94e72dc7f4d; no voters:
I20260619 22:20:53.126957 3569 raft_consensus.cc:2781] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:53.127040 3569 raft_consensus.cc:482] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:53.127095 3569 raft_consensus.cc:3037] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.128194 3569 raft_consensus.cc:504] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:53.128559 3569 leader_election.cc:288] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 election: Requested vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:53.128930 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf0ca38d77f4faa9bfb9fc984d856e0" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
I20260619 22:20:53.129012 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf0ca38d77f4faa9bfb9fc984d856e0" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d"
I20260619 22:20:53.129055 3213 raft_consensus.cc:3037] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.129150 3493 raft_consensus.cc:3037] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.130159 3213 raft_consensus.cc:2454] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 1.
I20260619 22:20:53.130465 3493 raft_consensus.cc:2454] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 1.
I20260619 22:20:53.130484 3286 leader_election.cc:302] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, a8c083b15f3b485b9cf921aa1cb042b2; no voters:
I20260619 22:20:53.130762 3569 raft_consensus.cc:2781] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:53.130996 3569 raft_consensus.cc:686] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 LEADER]: Becoming Leader. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Running, Role: LEADER
I20260619 22:20:53.131320 3569 consensus_queue.cc:237] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } }
I20260619 22:20:53.134531 3055 catalog_manager.cc:5495] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130). New cstate: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } health_report { overall_health: HEALTHY } } }
Jun 19 22:20:53 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907651, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:53.221069 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.212739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.1.74.190:40475)
0619 22:20:53.212861 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.212865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.213127 (+ 262us) server_negotiation.cc:407] Connection header received
0619 22:20:53.213299 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.213301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.213343 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.213423 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:53.214075 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.214591 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.215408 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.215620 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.215725 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.217996 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:53.218000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:53.218003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:53.218030 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:53.219330 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:53.219810 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:53.219820 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:53.219822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:53.219884 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:53.220144 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:53.220150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:53.220151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:53.220321 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:53.220438 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.220729 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.220919 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":49}
I20260619 22:20:53.221077 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.212714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40475 (local address 127.0.0.1:46102)
0619 22:20:53.213038 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.213050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.213183 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.213439 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.213442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.213869 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:20:53.213954 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.213960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.214695 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.214710 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.215294 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.215302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.215733 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.215736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.215859 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.216399 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:20:53.216410 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:20:53.217858 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:20:53.219433 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:53.219444 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:53.219451 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:53.219697 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:53.219977 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:20:53.219980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:20:53.219982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:20:53.220059 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:20:53.220436 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:20:53.220443 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:20:53.220565 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.220677 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.220870 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":18,"thread_start_us":117,"threads_started":1}
I20260619 22:20:53.225250 3055 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:46102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 22:20:53.225890 3055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:53.230728 3193 tablet_service.cc:1467] Processing CreateTablet for tablet 467d6288753e47d4b0fe53632e9f22bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dec8152cf76a4bceb91d0b9f9a4d678a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:20:53.230739 3333 tablet_service.cc:1467] Processing CreateTablet for tablet 467d6288753e47d4b0fe53632e9f22bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dec8152cf76a4bceb91d0b9f9a4d678a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:20:53.231539 3333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467d6288753e47d4b0fe53632e9f22bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:53.231544 3193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467d6288753e47d4b0fe53632e9f22bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:53.234189 3563 tablet_bootstrap.cc:492] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: Bootstrap starting.
I20260619 22:20:53.234186 3562 tablet_bootstrap.cc:492] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: Bootstrap starting.
I20260619 22:20:53.235236 3563 tablet_bootstrap.cc:654] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:53.234694 3473 tablet_service.cc:1467] Processing CreateTablet for tablet 467d6288753e47d4b0fe53632e9f22bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dec8152cf76a4bceb91d0b9f9a4d678a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:20:53.235491 3473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467d6288753e47d4b0fe53632e9f22bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:53.235500 3562 tablet_bootstrap.cc:654] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:53.236881 3562 tablet_bootstrap.cc:492] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: No bootstrap required, opened a new log
I20260619 22:20:53.236974 3562 ts_tablet_manager.cc:1397] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 22:20:53.237440 3562 raft_consensus.cc:348] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.237562 3562 raft_consensus.cc:374] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:53.237615 3562 raft_consensus.cc:729] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Initialized, Role: FOLLOWER
I20260619 22:20:53.237762 3562 consensus_queue.cc:260] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.238095 3562 ts_tablet_manager.cc:1428] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 22:20:53.238098 3569 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.238262 3569 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.238597 3564 tablet_bootstrap.cc:492] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: Bootstrap starting.
I20260619 22:20:53.239477 3563 tablet_bootstrap.cc:492] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: No bootstrap required, opened a new log
I20260619 22:20:53.239584 3563 ts_tablet_manager.cc:1397] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 22:20:53.239696 3564 tablet_bootstrap.cc:654] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:53.240042 3563 raft_consensus.cc:348] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.240151 3563 raft_consensus.cc:374] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:53.240190 3563 raft_consensus.cc:729] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c083b15f3b485b9cf921aa1cb042b2, State: Initialized, Role: FOLLOWER
I20260619 22:20:53.240329 3563 consensus_queue.cc:260] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.240872 3568 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.241003 3568 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.241024 3564 tablet_bootstrap.cc:492] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: No bootstrap required, opened a new log
I20260619 22:20:53.241119 3564 ts_tablet_manager.cc:1397] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 22:20:53.241258 3563 ts_tablet_manager.cc:1428] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:20:53.241570 3564 raft_consensus.cc:348] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.241679 3564 raft_consensus.cc:374] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:53.241714 3564 raft_consensus.cc:729] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8635171b39d042dfa798b94e72dc7f4d, State: Initialized, Role: FOLLOWER
I20260619 22:20:53.241858 3564 consensus_queue.cc:260] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.242148 3564 ts_tablet_manager.cc:1428] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:20:53.242137 3570 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.242271 3570 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.278088 3569 raft_consensus.cc:482] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:53.278223 3569 raft_consensus.cc:504] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.278669 3569 leader_election.cc:288] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:53.279241 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467d6288753e47d4b0fe53632e9f22bd" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" is_pre_election: true
I20260619 22:20:53.279280 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467d6288753e47d4b0fe53632e9f22bd" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d" is_pre_election: true
I20260619 22:20:53.279404 3493 raft_consensus.cc:2454] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 0.
I20260619 22:20:53.279405 3213 raft_consensus.cc:2454] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 0.
I20260619 22:20:53.279707 3286 leader_election.cc:302] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, 8635171b39d042dfa798b94e72dc7f4d; no voters:
I20260619 22:20:53.279872 3569 raft_consensus.cc:2781] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:53.279930 3569 raft_consensus.cc:482] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:53.279978 3569 raft_consensus.cc:3037] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.281128 3569 raft_consensus.cc:504] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.281495 3569 leader_election.cc:288] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 election: Requested vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:53.281896 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467d6288753e47d4b0fe53632e9f22bd" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
I20260619 22:20:53.281970 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467d6288753e47d4b0fe53632e9f22bd" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d"
I20260619 22:20:53.282029 3213 raft_consensus.cc:3037] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.282085 3493 raft_consensus.cc:3037] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.283205 3213 raft_consensus.cc:2454] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 1.
I20260619 22:20:53.283231 3493 raft_consensus.cc:2454] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 1.
I20260619 22:20:53.283476 3286 leader_election.cc:302] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, a8c083b15f3b485b9cf921aa1cb042b2; no voters:
I20260619 22:20:53.283689 3569 raft_consensus.cc:2781] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:53.283806 3569 raft_consensus.cc:686] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [term 1 LEADER]: Becoming Leader. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Running, Role: LEADER
I20260619 22:20:53.283972 3569 consensus_queue.cc:237] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.284655 3573 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: TxnStatusTablet state changed. Reason: New leader 62a19c3e7c174cd6bfebba1aade3234e. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.284773 3573 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:20:53.285080 3585 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 22:20:53.285319 3585 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 22:20:53.285684 3056 catalog_manager.cc:5495] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130). New cstate: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } health_report { overall_health: UNKNOWN } } }
I20260619 22:20:53.304865 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.300241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.0.0.1:41656)
0619 22:20:53.300416 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.300447 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.300866 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.301240 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.301242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.301249 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:53.301343 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.301348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.302346 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.302348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.303100 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.303110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.303662 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.303665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.303725 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.303770 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.304570 (+ 800us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.304584 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.304651 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.304718 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 22:20:53.304880 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.300367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.1.74.130:45893)
0619 22:20:53.300487 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.300492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.300677 (+ 185us) server_negotiation.cc:407] Connection header received
0619 22:20:53.301030 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.301033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.301108 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.301215 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:53.301444 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.302243 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.303222 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.303557 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.303658 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.303905 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.304436 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.304596 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.304663 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.304725 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 22:20:53.310227 3213 raft_consensus.cc:1261] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Refusing update from remote peer 62a19c3e7c174cd6bfebba1aade3234e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:53.310253 3493 raft_consensus.cc:1261] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Refusing update from remote peer 62a19c3e7c174cd6bfebba1aade3234e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:53.311039 3573 consensus_queue.cc:1035] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:53.311269 3569 consensus_queue.cc:1035] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:53.314822 3568 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: TxnStatusTablet state changed. Reason: New leader 62a19c3e7c174cd6bfebba1aade3234e. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.314996 3568 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.315164 3570 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: TxnStatusTablet state changed. Reason: New leader 62a19c3e7c174cd6bfebba1aade3234e. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.315268 3570 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.320734 3569 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.320859 3569 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:20:53.320952 3569 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.321028 3569 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 62a19c3e7c174cd6bfebba1aade3234e: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:20:53.321002 3570 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.321100 3588 mvcc.cc:205] Tried to move back new op lower bound from 7298693747951407104 to 7298693747852390400. Current Snapshot: MvccSnapshot[applied={T|T < 7298693747951407104}]
I20260619 22:20:53.321130 3570 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P 8635171b39d042dfa798b94e72dc7f4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.323271 3568 tablet_replica.cc:439] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } } }
I20260619 22:20:53.323385 3568 tablet_replica.cc:442] T 467d6288753e47d4b0fe53632e9f22bd P a8c083b15f3b485b9cf921aa1cb042b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:20:53.325297 3590 mvcc.cc:205] Tried to move back new op lower bound from 7298693747951407104 to 7298693747852390400. Current Snapshot: MvccSnapshot[applied={T|T < 7298693747951407104}]
I20260619 22:20:53.325873 3589 mvcc.cc:205] Tried to move back new op lower bound from 7298693747951407104 to 7298693747852390400. Current Snapshot: MvccSnapshot[applied={T|T < 7298693747951407104}]
I20260619 22:20:53.335203 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.331616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.1.74.130:45893)
0619 22:20:53.331753 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.331758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.331943 (+ 185us) server_negotiation.cc:407] Connection header received
0619 22:20:53.332044 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.332047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.332096 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.332175 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:53.332410 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.333177 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.334045 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.334395 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.334485 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.334763 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.334868 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.334940 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.335016 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.335065 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:20:53.347380 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.343427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:43459 (local address 127.1.74.130:45893)
0619 22:20:53.343623 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.343628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.343853 (+ 225us) server_negotiation.cc:407] Connection header received
0619 22:20:53.343979 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.343981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.344027 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.344097 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:53.344364 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.344937 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.346160 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.346364 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.346452 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.346908 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.346981 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.347052 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.347217 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.347270 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260619 22:20:53.347427 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.343330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.1.74.130:43459)
0619 22:20:53.343768 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.343782 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.343885 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.344132 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.344135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.344142 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:53.344268 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.344274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.345044 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.345047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.346023 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.346033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.346498 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.346502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.346603 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.346650 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.347076 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.347093 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.347166 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.347247 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":97,"threads_started":1}
I20260619 22:20:53.354144 3493 raft_consensus.cc:1261] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Refusing update from remote peer 62a19c3e7c174cd6bfebba1aade3234e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:53.354470 3213 raft_consensus.cc:1261] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Refusing update from remote peer 62a19c3e7c174cd6bfebba1aade3234e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:53.354552 3601 consensus_queue.cc:1035] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:53.354826 3601 consensus_queue.cc:1035] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:53.393546 3056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:53.393709 3056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:53.396257 3056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Sending DeleteTablet for 3 replicas of tablet 2bf0ca38d77f4faa9bfb9fc984d856e0
I20260619 22:20:53.397306 3473 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:53 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58060
I20260619 22:20:53.397421 3193 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:53 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 22:20:53.397816 3333 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:53 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:41644
I20260619 22:20:53.397840 3056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:20:53.398238 3056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:53.398281 3607 tablet_replica.cc:330] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: stopping tablet replica
I20260619 22:20:53.398643 3607 raft_consensus.cc:2229] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:20:53.399087 3606 tablet_replica.cc:330] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: stopping tablet replica
I20260619 22:20:53.399082 3607 raft_consensus.cc:2258] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:53.399461 3606 raft_consensus.cc:2229] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:53.399718 3606 raft_consensus.cc:2258] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:53.400403 3607 ts_tablet_manager.cc:1905] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:53.401078 3606 ts_tablet_manager.cc:1905] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:53.402999 3333 tablet_service.cc:1467] Processing CreateTablet for tablet 81ef902a7065464dbda54b7824a66c33 (DEFAULT_TABLE table=test-table [id=1b6f739bcec44a0e8772a791c1e8a111]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:53.403069 3605 tablet_replica.cc:330] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: stopping tablet replica
I20260619 22:20:53.403301 3333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ef902a7065464dbda54b7824a66c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:53.403589 3605 raft_consensus.cc:2229] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:53.403709 3607 ts_tablet_manager.cc:1918] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:53.403801 3607 log.cc:1196] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/2bf0ca38d77f4faa9bfb9fc984d856e0
I20260619 22:20:53.404119 3607 ts_tablet_manager.cc:1939] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting consensus metadata
I20260619 22:20:53.404346 3605 raft_consensus.cc:2258] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:53.405557 3562 tablet_bootstrap.cc:492] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Bootstrap starting.
I20260619 22:20:53.405716 3606 ts_tablet_manager.cc:1918] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:53.405843 3606 log.cc:1196] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/2bf0ca38d77f4faa9bfb9fc984d856e0
I20260619 22:20:53.406271 3606 ts_tablet_manager.cc:1939] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting consensus metadata
I20260619 22:20:53.406514 3562 tablet_bootstrap.cc:654] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:53.407297 3041 catalog_manager.cc:4841] TS 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893): tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 (table test-table [id=adbfa792175b40a9a138a2acd8439a95]) successfully deleted
I20260619 22:20:53.407691 3562 tablet_bootstrap.cc:492] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: No bootstrap required, opened a new log
I20260619 22:20:53.407801 3562 ts_tablet_manager.cc:1397] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 22:20:53.407871 3042 catalog_manager.cc:4841] TS a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527): tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 (table test-table [id=adbfa792175b40a9a138a2acd8439a95]) successfully deleted
I20260619 22:20:53.408279 3473 tablet_service.cc:1467] Processing CreateTablet for tablet 81ef902a7065464dbda54b7824a66c33 (DEFAULT_TABLE table=test-table [id=1b6f739bcec44a0e8772a791c1e8a111]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:53.408255 3562 raft_consensus.cc:348] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.408368 3562 raft_consensus.cc:374] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:53.408399 3562 raft_consensus.cc:729] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Initialized, Role: FOLLOWER
I20260619 22:20:53.408350 3193 tablet_service.cc:1467] Processing CreateTablet for tablet 81ef902a7065464dbda54b7824a66c33 (DEFAULT_TABLE table=test-table [id=1b6f739bcec44a0e8772a791c1e8a111]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:53.408545 3473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ef902a7065464dbda54b7824a66c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:53.408511 3562 consensus_queue.cc:260] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.408588 3193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ef902a7065464dbda54b7824a66c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:53.408772 3562 ts_tablet_manager.cc:1428] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 22:20:53.410941 3563 tablet_bootstrap.cc:492] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Bootstrap starting.
I20260619 22:20:53.411147 3564 tablet_bootstrap.cc:492] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Bootstrap starting.
I20260619 22:20:53.412003 3563 tablet_bootstrap.cc:654] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:53.412177 3564 tablet_bootstrap.cc:654] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:53.413466 3563 tablet_bootstrap.cc:492] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: No bootstrap required, opened a new log
I20260619 22:20:53.413551 3563 ts_tablet_manager.cc:1397] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 22:20:53.414008 3563 raft_consensus.cc:348] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.414143 3563 raft_consensus.cc:374] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:53.414175 3563 raft_consensus.cc:729] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c083b15f3b485b9cf921aa1cb042b2, State: Initialized, Role: FOLLOWER
I20260619 22:20:53.414312 3563 consensus_queue.cc:260] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.414633 3563 ts_tablet_manager.cc:1428] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 22:20:53.415874 3605 ts_tablet_manager.cc:1905] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:53.417083 3564 tablet_bootstrap.cc:492] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: No bootstrap required, opened a new log
I20260619 22:20:53.417189 3564 ts_tablet_manager.cc:1397] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 22:20:53.417693 3564 raft_consensus.cc:348] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.417824 3564 raft_consensus.cc:374] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:53.417863 3564 raft_consensus.cc:729] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8635171b39d042dfa798b94e72dc7f4d, State: Initialized, Role: FOLLOWER
I20260619 22:20:53.418011 3564 consensus_queue.cc:260] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.418972 3605 ts_tablet_manager.cc:1918] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:53.419085 3605 log.cc:1196] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/2bf0ca38d77f4faa9bfb9fc984d856e0
I20260619 22:20:53.419138 3564 ts_tablet_manager.cc:1428] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:53.419446 3605 ts_tablet_manager.cc:1939] T 2bf0ca38d77f4faa9bfb9fc984d856e0 P 8635171b39d042dfa798b94e72dc7f4d: Deleting consensus metadata
I20260619 22:20:53.420709 3042 catalog_manager.cc:4841] TS 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127): tablet 2bf0ca38d77f4faa9bfb9fc984d856e0 (table test-table [id=adbfa792175b40a9a138a2acd8439a95]) successfully deleted
I20260619 22:20:53.498659 3570 raft_consensus.cc:482] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:53.499094 3570 raft_consensus.cc:504] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.500362 3570 leader_election.cc:288] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893)
I20260619 22:20:53.505673 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.500764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:45805 (local address 127.1.74.130:45893)
0619 22:20:53.500914 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.500918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.501310 (+ 392us) server_negotiation.cc:407] Connection header received
0619 22:20:53.501501 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.501504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.501550 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.501597 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:53.502071 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.502883 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.504493 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.505023 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.505154 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.505210 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.505418 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.505528 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 22:20:53.505849 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.500796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49847 (local address 127.1.74.129:45527)
0619 22:20:53.500913 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.500918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.501182 (+ 264us) server_negotiation.cc:407] Connection header received
0619 22:20:53.501415 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.501419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.501472 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.501524 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:53.502029 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.502575 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.504467 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.505106 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.505224 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.505274 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.505636 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.505730 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:20:53.505867 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.500607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.1.74.131:45805)
0619 22:20:53.501217 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.501234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.501385 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.501735 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.501738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.501772 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:53.501946 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.501953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.502991 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.502994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.504323 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.504333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.505147 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.505150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.505266 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.505290 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.505398 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.505549 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":334,"thread_start_us":105,"threads_started":1}
I20260619 22:20:53.506390 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.500604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.1.74.131:49847)
0619 22:20:53.501074 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.501090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.501276 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.501671 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.501674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.501691 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:53.501894 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.501903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.502757 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.502760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.504323 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.504333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.505336 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.505340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.505481 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.505512 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.506169 (+ 657us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.506245 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":144,"thread_start_us":100,"threads_started":1}
I20260619 22:20:53.506569 3351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ef902a7065464dbda54b7824a66c33" candidate_uuid: "8635171b39d042dfa798b94e72dc7f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a19c3e7c174cd6bfebba1aade3234e" is_pre_election: true
I20260619 22:20:53.506804 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ef902a7065464dbda54b7824a66c33" candidate_uuid: "8635171b39d042dfa798b94e72dc7f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" is_pre_election: true
I20260619 22:20:53.506853 3351 raft_consensus.cc:2454] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8635171b39d042dfa798b94e72dc7f4d in term 0.
I20260619 22:20:53.506978 3212 raft_consensus.cc:2454] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8635171b39d042dfa798b94e72dc7f4d in term 0.
I20260619 22:20:53.507381 3426 leader_election.cc:302] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, 8635171b39d042dfa798b94e72dc7f4d, a8c083b15f3b485b9cf921aa1cb042b2; no voters:
I20260619 22:20:53.507663 3570 raft_consensus.cc:2781] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:53.507735 3570 raft_consensus.cc:482] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:53.507776 3570 raft_consensus.cc:3037] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.509044 3570 raft_consensus.cc:504] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.509442 3570 leader_election.cc:288] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [CANDIDATE]: Term 1 election: Requested vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893)
I20260619 22:20:53.509847 3351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ef902a7065464dbda54b7824a66c33" candidate_uuid: "8635171b39d042dfa798b94e72dc7f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a19c3e7c174cd6bfebba1aade3234e"
I20260619 22:20:53.509991 3351 raft_consensus.cc:3037] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.509959 3212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ef902a7065464dbda54b7824a66c33" candidate_uuid: "8635171b39d042dfa798b94e72dc7f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
I20260619 22:20:53.510095 3212 raft_consensus.cc:3037] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:53.511154 3351 raft_consensus.cc:2454] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8635171b39d042dfa798b94e72dc7f4d in term 1.
I20260619 22:20:53.511327 3212 raft_consensus.cc:2454] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8635171b39d042dfa798b94e72dc7f4d in term 1.
I20260619 22:20:53.511510 3425 leader_election.cc:302] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, 8635171b39d042dfa798b94e72dc7f4d; no voters:
I20260619 22:20:53.511682 3570 raft_consensus.cc:2781] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:53.511945 3570 raft_consensus.cc:686] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 1 LEADER]: Becoming Leader. State: Replica: 8635171b39d042dfa798b94e72dc7f4d, State: Running, Role: LEADER
I20260619 22:20:53.512246 3570 consensus_queue.cc:237] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:53.515187 3055 catalog_manager.cc:5495] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131). New cstate: current_term: 1 leader_uuid: "8635171b39d042dfa798b94e72dc7f4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } health_report { overall_health: HEALTHY } } }
I20260619 22:20:53.536394 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.532489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58070 (local address 127.1.74.131:35127)
0619 22:20:53.532617 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.532620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.532630 (+ 10us) server_negotiation.cc:407] Connection header received
0619 22:20:53.532699 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.532701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.532750 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.532874 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:53.533135 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.533661 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.534698 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.535005 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.535118 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.535393 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.536012 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.536158 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.536192 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.536248 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:20:53.548408 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.544773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.1.74.131:56699)
0619 22:20:53.545092 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.545114 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.545241 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.545505 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.545508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.545523 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:53.545658 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.545663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.546404 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.546409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.547133 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.547142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.547627 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.547631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.547733 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.547776 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.548094 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.548103 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.548170 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.548249 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":112,"threads_started":1}
I20260619 22:20:53.548378 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.544831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:56699 (local address 127.1.74.130:45893)
0619 22:20:53.544966 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.544972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.545192 (+ 220us) server_negotiation.cc:407] Connection header received
0619 22:20:53.545348 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.545352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.545402 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.545493 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:53.545762 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.546289 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.547253 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.547493 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.547614 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.547916 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.548001 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.548089 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.548180 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.548228 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 22:20:53.554955 3351 raft_consensus.cc:1261] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Refusing update from remote peer 8635171b39d042dfa798b94e72dc7f4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:53.555478 3212 raft_consensus.cc:1261] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Refusing update from remote peer 8635171b39d042dfa798b94e72dc7f4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:53.555598 3570 consensus_queue.cc:1035] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:53.555866 3613 consensus_queue.cc:1035] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:53.576642 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.572749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:35127 (local address 127.1.74.130:53905)
0619 22:20:53.572943 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:53.572960 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:53.573038 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:53.573307 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:53.573310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:53.573317 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:53.573423 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.573428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.574387 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.574390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.575276 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:53.575287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.575838 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.575841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:53.575932 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.575968 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.576341 (+ 373us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.576348 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 22:20:53.576419 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 22:20:53.576507 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 22:20:53.576643 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:53.572983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:53905 (local address 127.1.74.131:35127)
0619 22:20:53.573083 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:53.573089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:53.573104 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:20:53.573145 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:53.573147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:53.573199 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:53.573296 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:53.573540 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.574276 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.575427 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:53.575739 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:53.575842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:53.576153 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:53.576246 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:53.576317 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:53.576435 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 22:20:53.576492 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 22:20:53.577400 3589 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 22:20:55.142490 3055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:55.142654 3055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:55.145197 3055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Sending DeleteTablet for 3 replicas of tablet 81ef902a7065464dbda54b7824a66c33
I20260619 22:20:55.145810 3333 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ef902a7065464dbda54b7824a66c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:55 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:41644
I20260619 22:20:55.145805 3193 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ef902a7065464dbda54b7824a66c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:55 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 22:20:55.145844 3473 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ef902a7065464dbda54b7824a66c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:55 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58060
I20260619 22:20:55.146148 3646 tablet_replica.cc:330] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: stopping tablet replica
I20260619 22:20:55.146296 3646 raft_consensus.cc:2229] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:20:55.146307 3644 tablet_replica.cc:330] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: stopping tablet replica
I20260619 22:20:55.146452 3644 raft_consensus.cc:2229] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:55.146570 3644 raft_consensus.cc:2258] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:55.146588 3646 raft_consensus.cc:2258] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:55.147329 3644 ts_tablet_manager.cc:1905] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:55.148852 3645 tablet_replica.cc:330] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: stopping tablet replica
I20260619 22:20:55.148998 3645 raft_consensus.cc:2229] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:55.149129 3645 raft_consensus.cc:2258] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:55.149446 3646 ts_tablet_manager.cc:1905] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:55.149801 3645 ts_tablet_manager.cc:1905] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:55.150120 3644 ts_tablet_manager.cc:1918] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 22:20:55.150205 3644 log.cc:1196] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/81ef902a7065464dbda54b7824a66c33
I20260619 22:20:55.150465 3644 ts_tablet_manager.cc:1939] T 81ef902a7065464dbda54b7824a66c33 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting consensus metadata
I20260619 22:20:55.151324 3042 catalog_manager.cc:4841] TS a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527): tablet 81ef902a7065464dbda54b7824a66c33 (table test-table [id=1b6f739bcec44a0e8772a791c1e8a111]) successfully deleted
I20260619 22:20:55.152235 3645 ts_tablet_manager.cc:1918] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 22:20:55.152334 3645 log.cc:1196] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/81ef902a7065464dbda54b7824a66c33
I20260619 22:20:55.152577 3645 ts_tablet_manager.cc:1939] T 81ef902a7065464dbda54b7824a66c33 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting consensus metadata
I20260619 22:20:55.152827 3646 ts_tablet_manager.cc:1918] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 22:20:55.152907 3646 log.cc:1196] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/81ef902a7065464dbda54b7824a66c33
I20260619 22:20:55.153252 3646 ts_tablet_manager.cc:1939] T 81ef902a7065464dbda54b7824a66c33 P 8635171b39d042dfa798b94e72dc7f4d: Deleting consensus metadata
I20260619 22:20:55.153318 3041 catalog_manager.cc:4841] TS 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893): tablet 81ef902a7065464dbda54b7824a66c33 (table test-table [id=1b6f739bcec44a0e8772a791c1e8a111]) successfully deleted
I20260619 22:20:55.153944 3042 catalog_manager.cc:4841] TS 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127): tablet 81ef902a7065464dbda54b7824a66c33 (table test-table [id=1b6f739bcec44a0e8772a791c1e8a111]) successfully deleted
Jun 19 22:20:55 dist-test-slave-k84h krb5kdc[3011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907655, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 22:20:55 dist-test-slave-k84h krb5kdc[3011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907655, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:20:55.176108 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.166733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.1.74.190:40475)
0619 22:20:55.167057 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.167064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.167080 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:20:55.167275 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.167278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.167352 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.167445 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:20:55.168175 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.168741 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.169572 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.169801 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.169885 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.172676 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:20:55.172680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:20:55.172682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:20:55.172708 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:20:55.174281 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:55.174726 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:55.174730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:55.174732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:55.174823 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:20:55.175188 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:20:55.175193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:20:55.175195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:20:55.175393 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:20:55.175538 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.175780 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.175915 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":154,"threads_started":1}
I20260619 22:20:55.178690 3055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:20:55.179235 3055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:55.183962 3333 tablet_service.cc:1467] Processing CreateTablet for tablet 0f389016e43c42baa0274650b6caf782 (DEFAULT_TABLE table=test-table [id=7ae7e182a6c34dc1a7dc4fd65a7bcf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:55.183962 3193 tablet_service.cc:1467] Processing CreateTablet for tablet 0f389016e43c42baa0274650b6caf782 (DEFAULT_TABLE table=test-table [id=7ae7e182a6c34dc1a7dc4fd65a7bcf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:55.184008 3473 tablet_service.cc:1467] Processing CreateTablet for tablet 0f389016e43c42baa0274650b6caf782 (DEFAULT_TABLE table=test-table [id=7ae7e182a6c34dc1a7dc4fd65a7bcf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:55.184270 3193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f389016e43c42baa0274650b6caf782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:55.184270 3333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f389016e43c42baa0274650b6caf782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:55.184325 3473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f389016e43c42baa0274650b6caf782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:55.186944 3657 tablet_bootstrap.cc:492] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Bootstrap starting.
I20260619 22:20:55.187023 3655 tablet_bootstrap.cc:492] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Bootstrap starting.
I20260619 22:20:55.187108 3656 tablet_bootstrap.cc:492] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Bootstrap starting.
I20260619 22:20:55.187748 3657 tablet_bootstrap.cc:654] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:55.187906 3656 tablet_bootstrap.cc:654] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:55.187906 3655 tablet_bootstrap.cc:654] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:55.189064 3655 tablet_bootstrap.cc:492] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: No bootstrap required, opened a new log
I20260619 22:20:55.189064 3657 tablet_bootstrap.cc:492] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: No bootstrap required, opened a new log
I20260619 22:20:55.189149 3655 ts_tablet_manager.cc:1397] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:20:55.189149 3657 ts_tablet_manager.cc:1397] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:20:55.189574 3657 raft_consensus.cc:348] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.189574 3655 raft_consensus.cc:348] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.189694 3655 raft_consensus.cc:374] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:55.189694 3657 raft_consensus.cc:374] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:55.189728 3657 raft_consensus.cc:729] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Initialized, Role: FOLLOWER
I20260619 22:20:55.189728 3655 raft_consensus.cc:729] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8635171b39d042dfa798b94e72dc7f4d, State: Initialized, Role: FOLLOWER
I20260619 22:20:55.189872 3655 consensus_queue.cc:260] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.189872 3657 consensus_queue.cc:260] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.190133 3655 ts_tablet_manager.cc:1428] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:55.190133 3657 ts_tablet_manager.cc:1428] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:55.191473 3656 tablet_bootstrap.cc:492] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: No bootstrap required, opened a new log
I20260619 22:20:55.191546 3656 ts_tablet_manager.cc:1397] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 22:20:55.192057 3656 raft_consensus.cc:348] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.192149 3656 raft_consensus.cc:374] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:55.192193 3656 raft_consensus.cc:729] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c083b15f3b485b9cf921aa1cb042b2, State: Initialized, Role: FOLLOWER
I20260619 22:20:55.192345 3656 consensus_queue.cc:260] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.192942 3656 ts_tablet_manager.cc:1428] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:55.426460 3630 raft_consensus.cc:482] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:55.426605 3630 raft_consensus.cc:504] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.426985 3630 leader_election.cc:288] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:55.427552 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f389016e43c42baa0274650b6caf782" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d" is_pre_election: true
I20260619 22:20:55.427553 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f389016e43c42baa0274650b6caf782" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" is_pre_election: true
I20260619 22:20:55.427714 3493 raft_consensus.cc:2454] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 0.
I20260619 22:20:55.427714 3213 raft_consensus.cc:2454] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 0.
I20260619 22:20:55.428042 3286 leader_election.cc:302] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, 8635171b39d042dfa798b94e72dc7f4d; no voters:
I20260619 22:20:55.428224 3630 raft_consensus.cc:2781] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:55.428282 3630 raft_consensus.cc:482] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:55.428323 3630 raft_consensus.cc:3037] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:55.429324 3630 raft_consensus.cc:504] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.429643 3630 leader_election.cc:288] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 election: Requested vote from peers a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:55.430006 3213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f389016e43c42baa0274650b6caf782" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8c083b15f3b485b9cf921aa1cb042b2"
I20260619 22:20:55.430006 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f389016e43c42baa0274650b6caf782" candidate_uuid: "62a19c3e7c174cd6bfebba1aade3234e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d"
I20260619 22:20:55.430126 3213 raft_consensus.cc:3037] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:55.430135 3493 raft_consensus.cc:3037] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:55.431229 3213 raft_consensus.cc:2454] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 1.
I20260619 22:20:55.431229 3493 raft_consensus.cc:2454] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a19c3e7c174cd6bfebba1aade3234e in term 1.
I20260619 22:20:55.431550 3286 leader_election.cc:302] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, a8c083b15f3b485b9cf921aa1cb042b2; no voters:
I20260619 22:20:55.431715 3630 raft_consensus.cc:2781] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:55.431804 3630 raft_consensus.cc:686] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 LEADER]: Becoming Leader. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Running, Role: LEADER
I20260619 22:20:55.431951 3630 consensus_queue.cc:237] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.433588 3055 catalog_manager.cc:5495] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130). New cstate: current_term: 1 leader_uuid: "62a19c3e7c174cd6bfebba1aade3234e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } health_report { overall_health: UNKNOWN } } }
I20260619 22:20:55.490535 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.487113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.1.74.130:45893)
0619 22:20:55.487419 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.487430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.487447 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:20:55.487483 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.487484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.487525 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.487631 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:55.487878 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.488427 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.489301 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.489617 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.489748 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.490055 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:55.490146 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:55.490205 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.490304 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.490348 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
I20260619 22:20:55.497052 3493 raft_consensus.cc:1261] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Refusing update from remote peer 62a19c3e7c174cd6bfebba1aade3234e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:55.497052 3213 raft_consensus.cc:1261] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Refusing update from remote peer 62a19c3e7c174cd6bfebba1aade3234e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:55.497440 3625 consensus_queue.cc:1035] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:55.497565 3630 consensus_queue.cc:1035] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:55.528481 3056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:55.528676 3056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:55.530882 3056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Sending DeleteTablet for 3 replicas of tablet 0f389016e43c42baa0274650b6caf782
I20260619 22:20:55.531438 3332 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f389016e43c42baa0274650b6caf782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:55 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:41644
I20260619 22:20:55.531669 3193 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f389016e43c42baa0274650b6caf782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:55 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 22:20:55.531937 3645 tablet_replica.cc:330] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: stopping tablet replica
I20260619 22:20:55.532258 3473 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f389016e43c42baa0274650b6caf782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:55 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58060
I20260619 22:20:55.532480 3646 tablet_replica.cc:330] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: stopping tablet replica
I20260619 22:20:55.532600 3644 tablet_replica.cc:330] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: stopping tablet replica
I20260619 22:20:55.532755 3056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:20:55.533164 3056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:20:55.533219 3646 raft_consensus.cc:2229] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:55.533358 3646 raft_consensus.cc:2258] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:55.532891 3644 raft_consensus.cc:2229] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:55.533598 3644 raft_consensus.cc:2258] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:55.532979 3645 raft_consensus.cc:2229] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:20:55.533914 3645 raft_consensus.cc:2258] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:55.534669 3646 ts_tablet_manager.cc:1905] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:55.534669 3644 ts_tablet_manager.cc:1905] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:55.534690 3645 ts_tablet_manager.cc:1905] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:55.537256 3646 ts_tablet_manager.cc:1918] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:55.537331 3646 log.cc:1196] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/0f389016e43c42baa0274650b6caf782
I20260619 22:20:55.537361 3645 ts_tablet_manager.cc:1918] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:55.537429 3645 log.cc:1196] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/0f389016e43c42baa0274650b6caf782
I20260619 22:20:55.537609 3646 ts_tablet_manager.cc:1939] T 0f389016e43c42baa0274650b6caf782 P 8635171b39d042dfa798b94e72dc7f4d: Deleting consensus metadata
I20260619 22:20:55.537699 3645 ts_tablet_manager.cc:1939] T 0f389016e43c42baa0274650b6caf782 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting consensus metadata
I20260619 22:20:55.538293 3644 ts_tablet_manager.cc:1918] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:20:55.538296 3193 tablet_service.cc:1467] Processing CreateTablet for tablet b4c43b676c484b77b42f70d19c88bc55 (DEFAULT_TABLE table=test-table [id=17a969a2a2a749fa8a95d53b863b31fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:55.538443 3644 log.cc:1196] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/0f389016e43c42baa0274650b6caf782
I20260619 22:20:55.538605 3193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c43b676c484b77b42f70d19c88bc55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:55.538628 3473 tablet_service.cc:1467] Processing CreateTablet for tablet b4c43b676c484b77b42f70d19c88bc55 (DEFAULT_TABLE table=test-table [id=17a969a2a2a749fa8a95d53b863b31fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:55.538700 3644 ts_tablet_manager.cc:1939] T 0f389016e43c42baa0274650b6caf782 P a8c083b15f3b485b9cf921aa1cb042b2: Deleting consensus metadata
I20260619 22:20:55.538856 3473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c43b676c484b77b42f70d19c88bc55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:55.540485 3656 tablet_bootstrap.cc:492] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2: Bootstrap starting.
I20260619 22:20:55.540822 3655 tablet_bootstrap.cc:492] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Bootstrap starting.
I20260619 22:20:55.540853 3041 catalog_manager.cc:4841] TS 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893): tablet 0f389016e43c42baa0274650b6caf782 (table test-table [id=7ae7e182a6c34dc1a7dc4fd65a7bcf2f]) successfully deleted
I20260619 22:20:55.541230 3332 tablet_service.cc:1467] Processing CreateTablet for tablet b4c43b676c484b77b42f70d19c88bc55 (DEFAULT_TABLE table=test-table [id=17a969a2a2a749fa8a95d53b863b31fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:20:55.541483 3332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4c43b676c484b77b42f70d19c88bc55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:20:55.541494 3656 tablet_bootstrap.cc:654] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:55.541574 3655 tablet_bootstrap.cc:654] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:55.542086 3042 catalog_manager.cc:4841] TS a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527): tablet 0f389016e43c42baa0274650b6caf782 (table test-table [id=7ae7e182a6c34dc1a7dc4fd65a7bcf2f]) successfully deleted
I20260619 22:20:55.542822 3656 tablet_bootstrap.cc:492] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2: No bootstrap required, opened a new log
I20260619 22:20:55.542945 3656 ts_tablet_manager.cc:1397] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 22:20:55.542956 3042 catalog_manager.cc:4841] TS 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127): tablet 0f389016e43c42baa0274650b6caf782 (table test-table [id=7ae7e182a6c34dc1a7dc4fd65a7bcf2f]) successfully deleted
I20260619 22:20:55.543380 3656 raft_consensus.cc:348] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.543488 3656 raft_consensus.cc:374] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:55.543526 3656 raft_consensus.cc:729] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c083b15f3b485b9cf921aa1cb042b2, State: Initialized, Role: FOLLOWER
I20260619 22:20:55.543681 3656 consensus_queue.cc:260] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.543955 3656 ts_tablet_manager.cc:1428] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 22:20:55.544030 3657 tablet_bootstrap.cc:492] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: Bootstrap starting.
I20260619 22:20:55.544919 3657 tablet_bootstrap.cc:654] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: Neither blocks nor log segments found. Creating new log.
I20260619 22:20:55.546741 3657 tablet_bootstrap.cc:492] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: No bootstrap required, opened a new log
I20260619 22:20:55.546825 3657 ts_tablet_manager.cc:1397] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 22:20:55.547202 3657 raft_consensus.cc:348] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.547315 3657 raft_consensus.cc:374] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:55.547343 3657 raft_consensus.cc:729] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a19c3e7c174cd6bfebba1aade3234e, State: Initialized, Role: FOLLOWER
I20260619 22:20:55.547459 3657 consensus_queue.cc:260] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.547688 3657 ts_tablet_manager.cc:1428] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:20:55.551674 3655 tablet_bootstrap.cc:492] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: No bootstrap required, opened a new log
I20260619 22:20:55.551784 3655 ts_tablet_manager.cc:1397] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260619 22:20:55.552219 3655 raft_consensus.cc:348] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.552345 3655 raft_consensus.cc:374] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:20:55.552372 3655 raft_consensus.cc:729] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8635171b39d042dfa798b94e72dc7f4d, State: Initialized, Role: FOLLOWER
I20260619 22:20:55.552490 3655 consensus_queue.cc:260] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.552729 3655 ts_tablet_manager.cc:1428] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:20:55.613261 3661 raft_consensus.cc:482] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:20:55.613447 3661 raft_consensus.cc:504] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.614593 3661 leader_election.cc:288] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:55.618709 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.614859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:35127 (local address 127.1.74.129:50019)
0619 22:20:55.615208 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:55.615223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:55.615386 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:55.615700 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:55.615703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:55.615727 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:55.615876 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.615881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.616651 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.616656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.617519 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.617528 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.618207 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.618211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.618311 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.618337 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:20:55.618415 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 22:20:55.618494 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":43,"spinlock_wait_cycles":19712,"thread_start_us":111,"threads_started":1}
I20260619 22:20:55.618957 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.614830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.1.74.129:58755)
0619 22:20:55.615182 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:55.615197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:55.615394 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:55.615652 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:55.615657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:55.615673 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:20:55.615865 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.615871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.616601 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.616604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.617544 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.617553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.618479 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.618483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.618614 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.618638 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:20:55.618715 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 22:20:55.618790 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":88,"spinlock_wait_cycles":6144,"thread_start_us":87,"threads_started":1}
I20260619 22:20:55.618985 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.615057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:58755 (local address 127.1.74.130:45893)
0619 22:20:55.615189 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.615193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.615308 (+ 115us) server_negotiation.cc:407] Connection header received
0619 22:20:55.615511 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.615514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.615555 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.615615 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:55.615971 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.616496 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.617703 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.618332 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.618502 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.618552 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.618746 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.618829 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:20:55.618726 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.615003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:50019 (local address 127.1.74.131:35127)
0619 22:20:55.615217 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.615234 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.615335 (+ 101us) server_negotiation.cc:407] Connection header received
0619 22:20:55.615529 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.615531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.615570 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.615665 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:20:55.616001 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.616505 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.617642 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.618097 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.618223 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.618274 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.618492 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.618571 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260619 22:20:55.619364 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c43b676c484b77b42f70d19c88bc55" candidate_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d" is_pre_election: true
I20260619 22:20:55.619359 3351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c43b676c484b77b42f70d19c88bc55" candidate_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a19c3e7c174cd6bfebba1aade3234e" is_pre_election: true
I20260619 22:20:55.619516 3493 raft_consensus.cc:2454] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8c083b15f3b485b9cf921aa1cb042b2 in term 0.
I20260619 22:20:55.619517 3351 raft_consensus.cc:2454] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8c083b15f3b485b9cf921aa1cb042b2 in term 0.
I20260619 22:20:55.619877 3145 leader_election.cc:302] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62a19c3e7c174cd6bfebba1aade3234e, 8635171b39d042dfa798b94e72dc7f4d, a8c083b15f3b485b9cf921aa1cb042b2; no voters:
I20260619 22:20:55.620174 3661 raft_consensus.cc:2781] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:20:55.620229 3661 raft_consensus.cc:482] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:20:55.620278 3661 raft_consensus.cc:3037] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:55.621297 3661 raft_consensus.cc:504] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.621672 3661 leader_election.cc:288] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [CANDIDATE]: Term 1 election: Requested vote from peers 62a19c3e7c174cd6bfebba1aade3234e (127.1.74.130:45893), 8635171b39d042dfa798b94e72dc7f4d (127.1.74.131:35127)
I20260619 22:20:55.622046 3493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c43b676c484b77b42f70d19c88bc55" candidate_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8635171b39d042dfa798b94e72dc7f4d"
I20260619 22:20:55.622128 3351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4c43b676c484b77b42f70d19c88bc55" candidate_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a19c3e7c174cd6bfebba1aade3234e"
I20260619 22:20:55.622195 3493 raft_consensus.cc:3037] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:55.622228 3351 raft_consensus.cc:3037] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:20:55.623355 3493 raft_consensus.cc:2454] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c083b15f3b485b9cf921aa1cb042b2 in term 1.
I20260619 22:20:55.623538 3351 raft_consensus.cc:2454] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8c083b15f3b485b9cf921aa1cb042b2 in term 1.
I20260619 22:20:55.623667 3146 leader_election.cc:302] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8635171b39d042dfa798b94e72dc7f4d, a8c083b15f3b485b9cf921aa1cb042b2; no voters:
I20260619 22:20:55.623828 3661 raft_consensus.cc:2781] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:20:55.624090 3661 raft_consensus.cc:686] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [term 1 LEADER]: Becoming Leader. State: Replica: a8c083b15f3b485b9cf921aa1cb042b2, State: Running, Role: LEADER
I20260619 22:20:55.624359 3661 consensus_queue.cc:237] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } }
I20260619 22:20:55.627216 3056 catalog_manager.cc:5495] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8c083b15f3b485b9cf921aa1cb042b2" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 } health_report { overall_health: UNKNOWN } } }
I20260619 22:20:55.656812 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.652914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.1.74.129:45527)
0619 22:20:55.653187 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.653194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.653207 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:20:55.653279 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.653281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.653332 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.653441 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:55.653617 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.654494 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.655369 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.655627 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.655719 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.655965 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:55.656468 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:55.656563 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.656624 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.656675 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":110,"threads_started":1}
I20260619 22:20:55.667085 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.663814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:44399 (local address 127.1.74.130:45893)
0619 22:20:55.663969 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.663973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.664027 (+ 54us) server_negotiation.cc:407] Connection header received
0619 22:20:55.664138 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.664141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.664186 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.664252 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:55.664515 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.665064 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.665911 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.666184 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.666315 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.666593 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:55.666675 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:55.666749 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.666891 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.666940 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 22:20:55.667104 3679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.663660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.1.74.129:44399)
0619 22:20:55.663954 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:55.663967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:55.664052 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:55.664274 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:55.664276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:55.664282 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:55.664384 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.664388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.665174 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.665180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.665793 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.665801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.666324 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.666327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.666414 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.666453 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:55.666761 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:55.666769 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 22:20:55.666872 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 22:20:55.666949 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":85,"threads_started":1}
I20260619 22:20:55.672927 3493 raft_consensus.cc:1261] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Refusing update from remote peer a8c083b15f3b485b9cf921aa1cb042b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:55.672927 3351 raft_consensus.cc:1261] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Refusing update from remote peer a8c083b15f3b485b9cf921aa1cb042b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:20:55.673396 3661 consensus_queue.cc:1035] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8635171b39d042dfa798b94e72dc7f4d" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 35127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:55.673646 3674 consensus_queue.cc:1035] T b4c43b676c484b77b42f70d19c88bc55 P a8c083b15f3b485b9cf921aa1cb042b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a19c3e7c174cd6bfebba1aade3234e" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:20:55.691522 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.687776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.1.74.130:54493)
0619 22:20:55.688142 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:55.688157 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:20:55.688273 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:20:55.688451 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:20:55.688453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:20:55.688458 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:20:55.688555 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.688559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.689298 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.689301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.690189 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:20:55.690199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.690792 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.690796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:20:55.690917 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.690954 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:20:55.691246 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:20:55.691253 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 22:20:55.691312 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 22:20:55.691393 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":88,"threads_started":1}
I20260619 22:20:55.691520 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:55.687836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:54493 (local address 127.1.74.129:45527)
0619 22:20:55.687970 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:20:55.687975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:20:55.688228 (+ 253us) server_negotiation.cc:407] Connection header received
0619 22:20:55.688343 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:20:55.688345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:20:55.688389 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:20:55.688448 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:20:55.688623 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.689159 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.690352 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:20:55.690679 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:20:55.690796 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:20:55.691073 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:20:55.691156 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:20:55.691232 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 22:20:55.691322 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 22:20:55.691373 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 22:20:55.692293 3627 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 22:20:57.252111 3055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:57.252352 3055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:20:57.255246 3055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2b018f052bd4b95b6d052b28d3edfb6: Sending DeleteTablet for 3 replicas of tablet b4c43b676c484b77b42f70d19c88bc55
I20260619 22:20:57.255954 3193 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c43b676c484b77b42f70d19c88bc55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53872
I20260619 22:20:57.255993 3332 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c43b676c484b77b42f70d19c88bc55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:41644
I20260619 22:20:57.256039 3473 tablet_service.cc:1514] Processing DeleteTablet for tablet b4c43b676c484b77b42f70d19c88bc55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:20:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58060
I20260619 22:20:57.256453 3710 tablet_replica.cc:330] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: stopping tablet replica
I20260619 22:20:57.256500 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3128
I20260619 22:20:57.257262 3710 raft_consensus.cc:2229] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:57.257431 3710 raft_consensus.cc:2258] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:57.257715 3709 tablet_replica.cc:330] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: stopping tablet replica
I20260619 22:20:57.257874 3709 raft_consensus.cc:2229] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:20:57.258003 3709 raft_consensus.cc:2258] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:20:57.258632 3710 ts_tablet_manager.cc:1905] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:20:57.259323 3709 ts_tablet_manager.cc:1905] T b4c43b676c484b77b42f70d19c88bc55 P 62a19c3e7c174cd6bfebba1aade3234e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 22:20:57.265115 3042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:45527 (error 108)
I20260619 22:20:57.265327 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3268
I20260619 22:20:57.267349 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:57.266383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.0.0.1:53890)
0619 22:20:57.267173 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:57.267249 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":78,"threads_started":1}
W20260619 22:20:57.267522 3042 catalog_manager.cc:4584] TS a8c083b15f3b485b9cf921aa1cb042b2 (127.1.74.129:45527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4c43b676c484b77b42f70d19c88bc55: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45527: connect: Connection refused (error 111)
I20260619 22:20:57.320964 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:57.320607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.0.0.1:53900)
0619 22:20:57.320800 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:57.320874 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 22:20:57.321748 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:57.321547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45527 (local address 127.0.0.1:53912)
0619 22:20:57.321643 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:57.321692 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 22:20:57.328361 3710 ts_tablet_manager.cc:1918] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 22:20:57.328500 3710 log.cc:1196] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/b4c43b676c484b77b42f70d19c88bc55
I20260619 22:20:57.328778 3710 ts_tablet_manager.cc:1939] T b4c43b676c484b77b42f70d19c88bc55 P 8635171b39d042dfa798b94e72dc7f4d: Deleting consensus metadata
I20260619 22:20:57.331893 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3408
I20260619 22:20:57.332850 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:20:57.331975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45893 (local address 127.0.0.1:41684)
0619 22:20:57.332678 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 22:20:57.332745 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:45893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606}
I20260619 22:20:57.340459 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3024
2026-06-19T22:20:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6667 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:20:58 dist-test-slave-k84h krb5kdc[3715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:20:58 dist-test-slave-k84h krb5kdc[3715](info): set up 2 sockets
Jun 19 22:20:58 dist-test-slave-k84h krb5kdc[3715](info): commencing operation
krb5kdc: starting...
W20260619 22:20:59.794919 1322 mini_kdc.cc:121] Time spent starting KDC: real 2.431s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:20:59 dist-test-slave-k84h krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:20:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:20:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:20:59.917698 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42601
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:42601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:00.037127 3728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:00.037426 3728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:00.037468 3728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:00.041565 3728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:00.041625 3728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:00.041644 3728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:00.041661 3728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:00.041676 3728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:00.047201 3728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:42601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:00.048345 3728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:00.049454 3728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:00.058504 3733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:00.058519 3736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:00.058575 3728 server_base.cc:1034] running on GCE node
W20260619 22:21:00.058552 3734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:00.059466 3728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:00.060662 3728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:00.061834 3728 hybrid_clock.cc:648] HybridClock initialized: now 1781907660061800 us; error 48 us; skew 500 ppm
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:00.070276 3728 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:00.071341 3728 webserver.cc:460] Webserver started at http://127.1.74.190:37193/ using document root <none> and password file <none>
I20260619 22:21:00.071892 3728 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:00.071951 3728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:00.072113 3728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:00.074209 3728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "e38ee33257994125aa85948f210d1019"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "4e4dd0a7ce19b660b403a788e875871b"
server_key_iv: "c6581dc41c5a2323e5f76854a6174b66"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.074731 3728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "e38ee33257994125aa85948f210d1019"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "4e4dd0a7ce19b660b403a788e875871b"
server_key_iv: "c6581dc41c5a2323e5f76854a6174b66"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.078496 3728 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 22:21:00.081085 3743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.082532 3728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 22:21:00.082713 3728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "e38ee33257994125aa85948f210d1019"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "4e4dd0a7ce19b660b403a788e875871b"
server_key_iv: "c6581dc41c5a2323e5f76854a6174b66"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.082895 3728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:00.102094 3728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:00.105831 3728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:00.106050 3728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:00.113658 3728 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:42601
I20260619 22:21:00.113687 3795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:42601 every 8 connection(s)
I20260619 22:21:00.114653 3728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:00.117624 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:00.123735 3796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019: Bootstrap starting.
I20260619 22:21:00.124862 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3728
I20260619 22:21:00.124989 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:00.125311 1322 external_mini_cluster.cc:1428] Setting key 6467fa8de4339c4a9e298da2c25fad31
I20260619 22:21:00.127022 3796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:00.127826 3796 log.cc:824] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:00.129807 3796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019: No bootstrap required, opened a new log
I20260619 22:21:00.133659 3796 raft_consensus.cc:348] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38ee33257994125aa85948f210d1019" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42601 } }
I20260619 22:21:00.133908 3796 raft_consensus.cc:374] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:00.133971 3796 raft_consensus.cc:729] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38ee33257994125aa85948f210d1019, State: Initialized, Role: FOLLOWER
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:00.134493 3796 consensus_queue.cc:260] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38ee33257994125aa85948f210d1019" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42601 } }
I20260619 22:21:00.134660 3796 raft_consensus.cc:388] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:00.134738 3796 raft_consensus.cc:482] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:00.134893 3796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:00.136133 3796 raft_consensus.cc:504] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38ee33257994125aa85948f210d1019" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42601 } }
I20260619 22:21:00.136526 3796 leader_election.cc:302] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e38ee33257994125aa85948f210d1019; no voters:
I20260619 22:21:00.136911 3796 leader_election.cc:288] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:00.137035 3801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:00.137274 3801 raft_consensus.cc:686] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [term 1 LEADER]: Becoming Leader. State: Replica: e38ee33257994125aa85948f210d1019, State: Running, Role: LEADER
I20260619 22:21:00.137720 3801 consensus_queue.cc:237] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38ee33257994125aa85948f210d1019" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42601 } }
I20260619 22:21:00.138347 3796 sys_catalog.cc:564] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:00.139729 3803 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e38ee33257994125aa85948f210d1019. Latest consensus state: current_term: 1 leader_uuid: "e38ee33257994125aa85948f210d1019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38ee33257994125aa85948f210d1019" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42601 } } }
I20260619 22:21:00.139729 3802 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e38ee33257994125aa85948f210d1019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38ee33257994125aa85948f210d1019" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42601 } } }
I20260619 22:21:00.139878 3803 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:00.139979 3802 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:00.141234 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.126515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.1.74.190:42601)
0619 22:21:00.127022 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:00.127038 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:00.127094 (+ 56us) server_negotiation.cc:407] Connection header received
0619 22:21:00.127793 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:00.127812 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:00.128218 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:00.128561 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:00.129400 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.130455 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.131415 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.131920 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.132093 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.134950 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:00.134955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:00.134971 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:00.135024 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:00.137184 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.137774 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.137780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.137787 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.137932 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.138259 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.138262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.138263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.138780 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:00.139019 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:00.139740 (+ 721us) server_negotiation.cc:299] Negotiation successful
0619 22:21:00.140100 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":144,"threads_started":1}
W20260619 22:21:00.143510 3816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:21:00.143599 3816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:21:00.143659 3813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:00.144439 3813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:00.149958 3813 catalog_manager.cc:1347] Generated new cluster ID: e9ecd02fdcd34e1487ff36e7fd7c3ed4
I20260619 22:21:00.150041 3813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:00.168833 3813 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:00.170236 3813 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:00.176255 3813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e38ee33257994125aa85948f210d1019: Generated new TSK 0
I20260619 22:21:00.177234 3813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:00.210330 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42601
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:00.319815 3822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:00.320120 3822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:00.320202 3822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:00.324226 3822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:00.324308 3822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:00.324397 3822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:00.329532 3822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:00.330624 3822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:00.331691 3822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:00.341466 3830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:00.341486 3827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:00.341461 3828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:00.341588 3822 server_base.cc:1034] running on GCE node
I20260619 22:21:00.342213 3822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:00.342964 3822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:00.344152 3822 hybrid_clock.cc:648] HybridClock initialized: now 1781907660344137 us; error 35 us; skew 500 ppm
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:00.347208 3822 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:00.348145 3822 webserver.cc:460] Webserver started at http://127.1.74.129:36737/ using document root <none> and password file <none>
I20260619 22:21:00.348755 3822 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:00.348851 3822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:00.349089 3822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:00.351228 3822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "98fefce1d13d431bb75c06e0989c1e9c"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "1288d6550b97dd3ec2544e602cc56b9e"
server_key_iv: "ac7e8ce3e2bc48f93701810231fb796c"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.351801 3822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "98fefce1d13d431bb75c06e0989c1e9c"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "1288d6550b97dd3ec2544e602cc56b9e"
server_key_iv: "ac7e8ce3e2bc48f93701810231fb796c"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.355589 3822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:21:00.358116 3837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.359454 3822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:21:00.359587 3822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "98fefce1d13d431bb75c06e0989c1e9c"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "1288d6550b97dd3ec2544e602cc56b9e"
server_key_iv: "ac7e8ce3e2bc48f93701810231fb796c"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.359699 3822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:00.372087 3822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:00.375384 3822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:00.375587 3822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:00.376185 3822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:00.377082 3822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:00.377161 3822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.377233 3822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:00.377285 3822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.386323 3822 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45661
I20260619 22:21:00.386355 3950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45661 every 8 connection(s)
I20260619 22:21:00.387356 3822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:00.396909 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3822
I20260619 22:21:00.397053 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:00.397333 1322 external_mini_cluster.cc:1428] Setting key 38a2fc7f21bdf714e87e644a06ef41b4
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:00.402262 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.389429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:38161 (local address 127.1.74.190:42601)
0619 22:21:00.389574 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:00.389578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:00.390497 (+ 919us) server_negotiation.cc:407] Connection header received
0619 22:21:00.391981 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:00.391985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:00.392044 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:00.392150 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:00.393335 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.393867 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.395090 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.395335 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.395451 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.398191 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:00.398195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:00.398200 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:00.398229 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:00.399824 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.400496 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.400502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.400504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.400606 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.400949 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.400955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.400959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.401212 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:00.401332 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:00.401838 (+ 506us) server_negotiation.cc:299] Negotiation successful
0619 22:21:00.401993 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 22:21:00.403043 3953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.389840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42601 (local address 127.1.74.129:38161)
0619 22:21:00.390333 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:00.390378 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:00.391762 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:00.392324 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:00.392331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:00.392699 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:00.393178 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:00.393193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.393992 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.393996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:00.394969 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:00.394979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.395490 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.395495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:00.395678 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.396289 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:00.396304 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:00.398025 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:00.399946 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:00.399956 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:00.399976 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:00.400386 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:00.400702 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:00.400704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:00.400705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:00.400856 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:00.401338 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:00.401344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:00.401581 (+ 237us) client_negotiation.cc:769] Sending connection context
0619 22:21:00.401744 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 22:21:00.402018 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
I20260619 22:21:00.404423 3951 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42601
I20260619 22:21:00.404661 3951 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:00.405251 3951 heartbeater.cc:511] Master 127.1.74.190:42601 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:00.406963 3760 ts_manager.cc:194] Registered new tserver with Master: 98fefce1d13d431bb75c06e0989c1e9c (127.1.74.129:45661)
I20260619 22:21:00.408260 3760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:38161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:21:00.424345 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42601
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:00.536556 3956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:00.536847 3956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:00.536891 3956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:00.541416 3956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:00.541486 3956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:00.541546 3956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:00.547278 3956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:00.548416 3956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:00.549401 3956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:00.559243 3964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:00.559384 3956 server_base.cc:1034] running on GCE node
W20260619 22:21:00.559243 3962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:00.559264 3961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:00.559892 3956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:00.560608 3956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:00.561795 3956 hybrid_clock.cc:648] HybridClock initialized: now 1781907660561784 us; error 34 us; skew 500 ppm
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:00.564790 3956 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:00.565639 3956 webserver.cc:460] Webserver started at http://127.1.74.130:34311/ using document root <none> and password file <none>
I20260619 22:21:00.566203 3956 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:00.566313 3956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:00.566553 3956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:00.568640 3956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "9d25046f037a41f19f03550f855f88e2"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "c173c40262036314404baa5b63fcf366"
server_key_iv: "b0cd684b2f75b7daeec5868686695237"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.569195 3956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "9d25046f037a41f19f03550f855f88e2"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "c173c40262036314404baa5b63fcf366"
server_key_iv: "b0cd684b2f75b7daeec5868686695237"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.573029 3956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:00.575408 3971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.576560 3956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:00.576692 3956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "9d25046f037a41f19f03550f855f88e2"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "c173c40262036314404baa5b63fcf366"
server_key_iv: "b0cd684b2f75b7daeec5868686695237"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.576803 3956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:00.611640 3956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:00.614720 3956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:00.614965 3956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:00.615578 3956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:00.616415 3956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:00.616487 3956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.616559 3956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:00.616612 3956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.625921 3956 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:38455
I20260619 22:21:00.625939 4084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:38455 every 8 connection(s)
I20260619 22:21:00.626900 3956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:00.631306 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 3956
I20260619 22:21:00.631425 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:00.631713 1322 external_mini_cluster.cc:1428] Setting key eb59ee284829493e6a61807149d6d94c
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:21:00.642318 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.628813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:54435 (local address 127.1.74.190:42601)
0619 22:21:00.628952 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:00.628955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:00.629788 (+ 833us) server_negotiation.cc:407] Connection header received
0619 22:21:00.631222 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:00.631226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:00.631286 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:00.631414 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:00.632902 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.633669 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.634890 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.635110 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.635226 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.638223 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:00.638231 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:00.638234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:00.638263 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:00.640017 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.640645 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.640649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.640650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.640749 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.641117 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.641121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.641123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.641339 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:00.641458 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:00.642078 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 22:21:00.642182 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:21:00.643359 4087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.629186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42601 (local address 127.1.74.130:54435)
0619 22:21:00.629628 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:00.629671 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:00.630975 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:00.631632 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:00.631639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:00.632075 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:00.632733 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:00.632752 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.633802 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.633805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:00.634738 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:00.634747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.635266 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.635270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:00.635454 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.636077 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:00.636093 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:00.637855 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:00.640147 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:00.640154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:00.640173 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:00.640533 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:00.640854 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:00.640857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:00.640859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:00.641013 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:00.641488 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:00.641495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:00.641835 (+ 340us) client_negotiation.cc:769] Sending connection context
0619 22:21:00.642026 (+ 191us) client_negotiation.cc:240] Negotiation successful
0619 22:21:00.642295 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:21:00.644639 4085 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42601
I20260619 22:21:00.644922 4085 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:00.645550 4085 heartbeater.cc:511] Master 127.1.74.190:42601 requested a full tablet report, sending...
I20260619 22:21:00.646744 3760 ts_manager.cc:194] Registered new tserver with Master: 9d25046f037a41f19f03550f855f88e2 (127.1.74.130:38455)
I20260619 22:21:00.647671 3760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:54435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:00.661574 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42601
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:00.776804 4090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:00.777086 4090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:00.777128 4090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:00.781314 4090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:00.781383 4090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:00.781478 4090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:00.786691 4090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:00.787765 4090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:00.788805 4090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:00.798048 4095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:00.798035 4098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:00.798040 4096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:00.798394 4090 server_base.cc:1034] running on GCE node
I20260619 22:21:00.798776 4090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:00.799439 4090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:00.800595 4090 hybrid_clock.cc:648] HybridClock initialized: now 1781907660800583 us; error 24 us; skew 500 ppm
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:00.803378 4090 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:00.804193 4090 webserver.cc:460] Webserver started at http://127.1.74.131:40859/ using document root <none> and password file <none>
I20260619 22:21:00.804847 4090 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:00.804934 4090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:00.805128 4090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:00.807183 4090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "f99f61766aaa439786274cff30efbfb2"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "c4df85a330545df689d0023fde9ae2c6"
server_key_iv: "e852c41c284f83a737279942ed04e922"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.807752 4090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "f99f61766aaa439786274cff30efbfb2"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "c4df85a330545df689d0023fde9ae2c6"
server_key_iv: "e852c41c284f83a737279942ed04e922"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.811368 4090 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:21:00.813794 4105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.815101 4090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:00.815228 4090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "f99f61766aaa439786274cff30efbfb2"
format_stamp: "Formatted at 2026-06-19 22:21:00 on dist-test-slave-k84h"
server_key: "c4df85a330545df689d0023fde9ae2c6"
server_key_iv: "e852c41c284f83a737279942ed04e922"
server_key_version: "encryptionkey@0"
I20260619 22:21:00.815368 4090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:00.830096 4090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:00.833318 4090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:00.833534 4090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:00.834117 4090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:00.834985 4090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:00.835052 4090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.835124 4090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:00.835175 4090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:00.844655 4090 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:44641
I20260619 22:21:00.844683 4218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:44641 every 8 connection(s)
I20260619 22:21:00.845625 4090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:00.848451 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4090
I20260619 22:21:00.848577 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:00.848872 1322 external_mini_cluster.cc:1428] Setting key eef5af891a7e77dca3fa2815f4b0c8ec
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[3715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907660, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:00.860737 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.847494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:34565 (local address 127.1.74.190:42601)
0619 22:21:00.847666 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:00.847671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:00.848592 (+ 921us) server_negotiation.cc:407] Connection header received
0619 22:21:00.850317 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:00.850321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:00.850374 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:00.850465 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:00.851772 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.852309 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.853479 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.853700 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.853811 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.856731 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:00.856734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:00.856736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:00.856763 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:00.858109 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.858744 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.858747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.858748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.858927 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:00.859408 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:00.859411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:00.859414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:00.859658 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:00.859785 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:00.860384 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 22:21:00.860529 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 22:21:00.861698 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:00.847871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42601 (local address 127.1.74.131:34565)
0619 22:21:00.848416 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:00.848478 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:00.850075 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:00.850644 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:00.850651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:00.851138 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:00.851627 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:00.851641 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.852451 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.852454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:00.853351 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:00.853360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:00.853848 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:00.853852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:00.854023 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:00.854580 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:00.854597 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:00.856527 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:00.858238 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:00.858243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:00.858258 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:00.858560 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:00.859058 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:00.859064 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:00.859066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:00.859275 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:00.859782 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:00.859790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:00.860069 (+ 279us) client_negotiation.cc:769] Sending connection context
0619 22:21:00.860321 (+ 252us) client_negotiation.cc:240] Negotiation successful
0619 22:21:00.860689 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":111,"threads_started":1}
I20260619 22:21:00.863320 4219 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42601
I20260619 22:21:00.863554 4219 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:00.864027 4219 heartbeater.cc:511] Master 127.1.74.190:42601 requested a full tablet report, sending...
I20260619 22:21:00.865226 3759 ts_manager.cc:194] Registered new tserver with Master: f99f61766aaa439786274cff30efbfb2 (127.1.74.131:44641)
I20260619 22:21:00.865800 3759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:34565
I20260619 22:21:00.875402 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 22:21:00.888913 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:00.887375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.1.74.190:42601)
0619 22:21:00.887561 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:00.887571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:00.887657 (+ 86us) server_negotiation.cc:407] Connection header received
0619 22:21:00.887798 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:00.887800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:00.887844 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:00.887916 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:00.888748 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44998: BlockingRecv error: recv got EOF from 127.0.0.1:44998 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 22:21:00.889940 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3822
I20260619 22:21:00.896205 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3956
I20260619 22:21:00.901814 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4090
I20260619 22:21:00.907184 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 3728
2026-06-19T22:21:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3565 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[4231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[4231](info): set up 2 sockets
Jun 19 22:21:00 dist-test-slave-k84h krb5kdc[4231](info): commencing operation
krb5kdc: starting...
W20260619 22:21:01.947081 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:02 dist-test-slave-k84h krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:02.074316 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:36083
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:36083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:02.188184 4244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:02.188519 4244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:02.188596 4244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:02.192808 4244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:02.192904 4244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:02.192945 4244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:02.193001 4244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:02.193040 4244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:02.198491 4244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:36083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:36083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:02.199733 4244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:02.200862 4244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:02.210505 4250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:02.210505 4249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:02.210538 4252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:02.210875 4244 server_base.cc:1034] running on GCE node
I20260619 22:21:02.211453 4244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:02.212647 4244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:02.213842 4244 hybrid_clock.cc:648] HybridClock initialized: now 1781907662213826 us; error 31 us; skew 500 ppm
Jun 19 22:21:02 dist-test-slave-k84h krb5kdc[4231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907662, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:02.216663 4244 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:02.217520 4244 webserver.cc:460] Webserver started at http://127.1.74.190:42425/ using document root <none> and password file <none>
I20260619 22:21:02.218040 4244 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:02.218097 4244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:02.218263 4244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:02.220391 4244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "496f30b3cd464eccb06693c7bcf6473d"
format_stamp: "Formatted at 2026-06-19 22:21:02 on dist-test-slave-k84h"
server_key: "6d761d8b213b77be271605790b35ff58"
server_key_iv: "2d5a388e2f390a84cfe5ba5c9e95277c"
server_key_version: "encryptionkey@0"
I20260619 22:21:02.220916 4244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "496f30b3cd464eccb06693c7bcf6473d"
format_stamp: "Formatted at 2026-06-19 22:21:02 on dist-test-slave-k84h"
server_key: "6d761d8b213b77be271605790b35ff58"
server_key_iv: "2d5a388e2f390a84cfe5ba5c9e95277c"
server_key_version: "encryptionkey@0"
I20260619 22:21:02.225248 4244 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:02.227830 4259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:02.229084 4244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:02.229239 4244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "496f30b3cd464eccb06693c7bcf6473d"
format_stamp: "Formatted at 2026-06-19 22:21:02 on dist-test-slave-k84h"
server_key: "6d761d8b213b77be271605790b35ff58"
server_key_iv: "2d5a388e2f390a84cfe5ba5c9e95277c"
server_key_version: "encryptionkey@0"
I20260619 22:21:02.229365 4244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:02.251811 4244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:02.255012 4244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:02.255219 4244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:02.262060 4244 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:36083
I20260619 22:21:02.262368 4311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:36083 every 8 connection(s)
I20260619 22:21:02.263093 4244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:02.265841 4312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:02.270983 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4244
I20260619 22:21:02.271103 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:02.271245 4312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d: Bootstrap starting.
I20260619 22:21:02.271376 1322 external_mini_cluster.cc:1428] Setting key 475c37a10b115d940d3c2f53211fd572
I20260619 22:21:02.273932 4312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:02.274799 4312 log.cc:824] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:02.277009 4312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d: No bootstrap required, opened a new log
Jun 19 22:21:02 dist-test-slave-k84h krb5kdc[4231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:02.280956 4312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496f30b3cd464eccb06693c7bcf6473d" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36083 } }
I20260619 22:21:02.281231 4312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:02.281301 4312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496f30b3cd464eccb06693c7bcf6473d, State: Initialized, Role: FOLLOWER
I20260619 22:21:02.281817 4312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496f30b3cd464eccb06693c7bcf6473d" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36083 } }
I20260619 22:21:02.282022 4312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:02.282104 4312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:02.282243 4312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:02.283190 4312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496f30b3cd464eccb06693c7bcf6473d" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36083 } }
I20260619 22:21:02.283511 4312 leader_election.cc:302] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 496f30b3cd464eccb06693c7bcf6473d; no voters:
I20260619 22:21:02.283849 4312 leader_election.cc:288] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:02.283895 4317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:02.284093 4317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [term 1 LEADER]: Becoming Leader. State: Replica: 496f30b3cd464eccb06693c7bcf6473d, State: Running, Role: LEADER
I20260619 22:21:02.284451 4317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496f30b3cd464eccb06693c7bcf6473d" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36083 } }
I20260619 22:21:02.285130 4312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:02.286398 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:02.272714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.1.74.190:36083)
0619 22:21:02.273142 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:02.273162 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:02.273202 (+ 40us) server_negotiation.cc:407] Connection header received
0619 22:21:02.274085 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:02.274108 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:02.274499 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:02.274933 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:02.275771 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:02.277116 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:02.277968 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:02.278637 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:02.278815 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:02.281118 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:02.281124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:02.281138 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:02.281169 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:02.283113 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:02.283593 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:02.283598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:02.283602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:02.283745 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:02.284044 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:02.284047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:02.284048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:02.284629 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:02.284807 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:02.285203 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 22:21:02.285506 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":122,"threads_started":1}
I20260619 22:21:02.287986 4318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "496f30b3cd464eccb06693c7bcf6473d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496f30b3cd464eccb06693c7bcf6473d" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36083 } } }
I20260619 22:21:02.288031 4319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 496f30b3cd464eccb06693c7bcf6473d. Latest consensus state: current_term: 1 leader_uuid: "496f30b3cd464eccb06693c7bcf6473d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "496f30b3cd464eccb06693c7bcf6473d" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36083 } } }
I20260619 22:21:02.288125 4318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:02.288134 4319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:02.289273 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 22:21:02.290982 4332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:21:02.291057 4332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:21:02.291136 4333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:02.291911 4333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:02.298048 4333 catalog_manager.cc:1347] Generated new cluster ID: 040730a75a6b4997b4ec07a22eb36f4f
I20260619 22:21:02.298123 4333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:02.309713 4333 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:02.310817 4333 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:02.312641 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:02.306698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.1.74.190:36083)
0619 22:21:02.306816 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:02.306821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:02.307128 (+ 307us) server_negotiation.cc:407] Connection header received
0619 22:21:02.307246 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:02.307250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:02.307308 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:02.307410 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:02.308003 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:02.308806 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:02.309937 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:02.310338 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:02.310456 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:02.311471 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:02.311477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:02.311482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:02.312086 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:02.312124 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:02.312146 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:02.312258 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:02.312389 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 22:21:02.312475 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":45}
I20260619 22:21:02.331787 4333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 496f30b3cd464eccb06693c7bcf6473d: Generated new TSK 0
I20260619 22:21:02.332607 4333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:02.338981 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4244
2026-06-19T22:21:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1428 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:02 dist-test-slave-k84h krb5kdc[4345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:02 dist-test-slave-k84h krb5kdc[4345](info): set up 2 sockets
Jun 19 22:21:02 dist-test-slave-k84h krb5kdc[4345](info): commencing operation
krb5kdc: starting...
W20260619 22:21:03.379019 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:03 dist-test-slave-k84h krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:03.510102 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37805
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:03.623940 4358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:03.624303 4358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:03.624387 4358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:03.628641 4358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:03.628744 4358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:03.628782 4358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:03.628821 4358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:03.628852 4358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:03.634380 4358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:03.635609 4358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:03.636689 4358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:03.645780 4364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:03.645756 4366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:03.645754 4363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:03.645829 4358 server_base.cc:1034] running on GCE node
I20260619 22:21:03.646682 4358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:03.647914 4358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:03.649120 4358 hybrid_clock.cc:648] HybridClock initialized: now 1781907663649087 us; error 50 us; skew 500 ppm
Jun 19 22:21:03 dist-test-slave-k84h krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:03.652158 4358 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:03.653067 4358 webserver.cc:460] Webserver started at http://127.1.74.190:39893/ using document root <none> and password file <none>
I20260619 22:21:03.653616 4358 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:03.653676 4358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:03.653855 4358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:03.655994 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "5993266752c0440388f0eece55261b44"
format_stamp: "Formatted at 2026-06-19 22:21:03 on dist-test-slave-k84h"
server_key: "a468c48f2cc0a324eed3f7257f932b23"
server_key_iv: "8c1d8d91c1b17e175a700a676ea550a3"
server_key_version: "encryptionkey@0"
I20260619 22:21:03.656600 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "5993266752c0440388f0eece55261b44"
format_stamp: "Formatted at 2026-06-19 22:21:03 on dist-test-slave-k84h"
server_key: "a468c48f2cc0a324eed3f7257f932b23"
server_key_iv: "8c1d8d91c1b17e175a700a676ea550a3"
server_key_version: "encryptionkey@0"
I20260619 22:21:03.660488 4358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 22:21:03.663110 4373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:03.664390 4358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:03.664534 4358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "5993266752c0440388f0eece55261b44"
format_stamp: "Formatted at 2026-06-19 22:21:03 on dist-test-slave-k84h"
server_key: "a468c48f2cc0a324eed3f7257f932b23"
server_key_iv: "8c1d8d91c1b17e175a700a676ea550a3"
server_key_version: "encryptionkey@0"
I20260619 22:21:03.664651 4358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:03.680208 4358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:03.683712 4358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:03.683933 4358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:03.691603 4358 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:37805
I20260619 22:21:03.691910 4425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:37805 every 8 connection(s)
I20260619 22:21:03.692765 4358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:03.695696 4426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:03.697314 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4358
I20260619 22:21:03.697408 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:03.697688 1322 external_mini_cluster.cc:1428] Setting key 8e42eea506ea890ec4f9dd0f55b90109
I20260619 22:21:03.702729 4426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44: Bootstrap starting.
I20260619 22:21:03.705507 4426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:03.706311 4426 log.cc:824] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:21:03 dist-test-slave-k84h krb5kdc[4345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:03.708787 4426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44: No bootstrap required, opened a new log
I20260619 22:21:03.713999 4426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5993266752c0440388f0eece55261b44" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37805 } }
I20260619 22:21:03.714287 4426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:03.714337 4426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5993266752c0440388f0eece55261b44, State: Initialized, Role: FOLLOWER
I20260619 22:21:03.714389 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:03.699281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.1.74.190:37805)
0619 22:21:03.699736 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:03.699758 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:03.699803 (+ 45us) server_negotiation.cc:407] Connection header received
0619 22:21:03.700679 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:03.700706 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:03.701160 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:03.701563 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:03.702295 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:03.703763 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:03.704705 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:03.705399 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:03.705588 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:03.708726 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:03.708732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:03.708749 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:03.708781 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:03.710919 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:03.711373 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:03.711379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:03.711385 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:03.711565 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:03.711950 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:03.711953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:03.711955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:03.712511 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:03.712760 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:03.713045 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 22:21:03.713379 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":123,"threads_started":1}
I20260619 22:21:03.714896 4426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5993266752c0440388f0eece55261b44" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37805 } }
I20260619 22:21:03.715071 4426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:03.715127 4426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:03.715263 4426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:03.716584 4426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5993266752c0440388f0eece55261b44" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37805 } }
I20260619 22:21:03.716979 4426 leader_election.cc:302] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5993266752c0440388f0eece55261b44; no voters:
I20260619 22:21:03.717303 4426 leader_election.cc:288] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:03.717419 4431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:03.717657 4431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [term 1 LEADER]: Becoming Leader. State: Replica: 5993266752c0440388f0eece55261b44, State: Running, Role: LEADER
I20260619 22:21:03.718017 4431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5993266752c0440388f0eece55261b44" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37805 } }
I20260619 22:21:03.718506 4426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:03.719854 4433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5993266752c0440388f0eece55261b44. Latest consensus state: current_term: 1 leader_uuid: "5993266752c0440388f0eece55261b44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5993266752c0440388f0eece55261b44" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37805 } } }
I20260619 22:21:03.719869 4432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5993266752c0440388f0eece55261b44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5993266752c0440388f0eece55261b44" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37805 } } }
I20260619 22:21:03.719980 4433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:03.719996 4432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:03.721035 4440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:03.723738 4440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:03.729027 4440 catalog_manager.cc:1347] Generated new cluster ID: 8af91b73302945de8c3714a83987cc0b
I20260619 22:21:03.729108 4440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:03.739435 4440 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:03.740372 4440 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:03.752425 4440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5993266752c0440388f0eece55261b44: Generated new TSK 0
I20260619 22:21:03.753127 4440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:03.791805 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37805
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:03.914716 4452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:03.915035 4452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:03.915082 4452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:03.920101 4452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:03.920176 4452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:03.920240 4452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:03.925935 4452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:03.927155 4452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:03.928411 4452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:03.939219 4460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:03.939219 4457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:03.939242 4458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:03.940008 4452 server_base.cc:1034] running on GCE node
I20260619 22:21:03.940506 4452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:03.941149 4452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:03.942315 4452 hybrid_clock.cc:648] HybridClock initialized: now 1781907663942296 us; error 34 us; skew 500 ppm
Jun 19 22:21:03 dist-test-slave-k84h krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:03.945669 4452 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:03.946952 4452 webserver.cc:460] Webserver started at http://127.1.74.129:36609/ using document root <none> and password file <none>
I20260619 22:21:03.947585 4452 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:03.947698 4452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:03.947911 4452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:03.950024 4452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "a0c4456c973e4d6bb2d3b3d5e786820c"
format_stamp: "Formatted at 2026-06-19 22:21:03 on dist-test-slave-k84h"
server_key: "df7fb51b7da69d64351dc36185a74972"
server_key_iv: "193e268a99340bc03187d4de7e756f01"
server_key_version: "encryptionkey@0"
I20260619 22:21:03.950548 4452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "a0c4456c973e4d6bb2d3b3d5e786820c"
format_stamp: "Formatted at 2026-06-19 22:21:03 on dist-test-slave-k84h"
server_key: "df7fb51b7da69d64351dc36185a74972"
server_key_iv: "193e268a99340bc03187d4de7e756f01"
server_key_version: "encryptionkey@0"
I20260619 22:21:03.954455 4452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 22:21:03.957736 4467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:03.959360 4452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 22:21:03.959471 4452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "a0c4456c973e4d6bb2d3b3d5e786820c"
format_stamp: "Formatted at 2026-06-19 22:21:03 on dist-test-slave-k84h"
server_key: "df7fb51b7da69d64351dc36185a74972"
server_key_iv: "193e268a99340bc03187d4de7e756f01"
server_key_version: "encryptionkey@0"
I20260619 22:21:03.959631 4452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:03.983659 4452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:03.987290 4452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:03.987543 4452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:03.988320 4452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:03.989280 4452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:03.989329 4452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:03.989403 4452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:03.989432 4452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:03.999323 4452 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:36653
I20260619 22:21:03.999341 4580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:36653 every 8 connection(s)
I20260619 22:21:04.000310 4452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:04.008977 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4452
I20260619 22:21:04.009088 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:04.009402 1322 external_mini_cluster.cc:1428] Setting key f5559f31578cb74e1f37e94baf8d6358
Jun 19 22:21:04 dist-test-slave-k84h krb5kdc[4345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:04.016641 4429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:04.002342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:51269 (local address 127.1.74.190:37805)
0619 22:21:04.002510 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:04.002514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:04.003463 (+ 949us) server_negotiation.cc:407] Connection header received
0619 22:21:04.004942 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:04.004947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:04.005018 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:04.005104 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:04.006389 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:04.007027 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:04.008269 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:04.008590 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:04.008795 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:04.012406 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:04.012409 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:04.012411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:04.012452 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:04.014183 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:04.014781 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:04.014785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:04.014787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:04.014931 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:04.015392 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:04.015397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:04.015398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:04.015607 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:04.015724 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:04.016294 (+ 570us) server_negotiation.cc:299] Negotiation successful
0619 22:21:04.016460 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 22:21:04.017542 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:04.002746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37805 (local address 127.1.74.129:51269)
0619 22:21:04.003292 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:04.003339 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:04.004694 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:04.005303 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:04.005311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:04.005711 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:04.006223 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:04.006238 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:04.007171 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:04.007174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:04.008124 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:04.008134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:04.009159 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:04.009169 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:04.009502 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:04.010510 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:04.010532 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:04.012207 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:04.014317 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:04.014325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:04.014345 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:04.014656 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:04.015032 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:04.015036 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:04.015037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:04.015286 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:04.015716 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:04.015723 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:04.016028 (+ 305us) client_negotiation.cc:769] Sending connection context
0619 22:21:04.016255 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 22:21:04.016624 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":129,"threads_started":1}
I20260619 22:21:04.019114 4581 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37805
I20260619 22:21:04.019424 4581 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:04.020109 4581 heartbeater.cc:511] Master 127.1.74.190:37805 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:04.021821 4390 ts_manager.cc:194] Registered new tserver with Master: a0c4456c973e4d6bb2d3b3d5e786820c (127.1.74.129:36653)
I20260619 22:21:04.023111 4390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:51269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
W20260619 22:21:04.873458 1322 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.74.130: real 0.862s user 0.001s sys 0.002s
I20260619 22:21:04.873966 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37805
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:04.989264 4586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:04.989640 4586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:04.989683 4586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:04.993603 4586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:04.993659 4586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:04.993716 4586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:04.998870 4586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:05.000159 4586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:05.001433 4586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:05.012056 4594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:05.012045 4592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:05.012049 4591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:05.012389 4586 server_base.cc:1034] running on GCE node
I20260619 22:21:05.012976 4586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:05.013695 4586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:05.014945 4586 hybrid_clock.cc:648] HybridClock initialized: now 1781907665014919 us; error 37 us; skew 500 ppm
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907665, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:05.017944 4586 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:05.019008 4586 webserver.cc:460] Webserver started at http://127.1.74.130:45511/ using document root <none> and password file <none>
I20260619 22:21:05.019711 4586 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:05.019909 4586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:05.020182 4586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:05.022387 4586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "519ac758f35a480cb95fd226e92ba4e6"
format_stamp: "Formatted at 2026-06-19 22:21:05 on dist-test-slave-k84h"
server_key: "d6143883d86e61c1e3f8715127794be6"
server_key_iv: "2352a35a59e9d7e0e491e976b90fe45e"
server_key_version: "encryptionkey@0"
I20260619 22:21:05.023051 4586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "519ac758f35a480cb95fd226e92ba4e6"
format_stamp: "Formatted at 2026-06-19 22:21:05 on dist-test-slave-k84h"
server_key: "d6143883d86e61c1e3f8715127794be6"
server_key_iv: "2352a35a59e9d7e0e491e976b90fe45e"
server_key_version: "encryptionkey@0"
I20260619 22:21:05.025593 4581 heartbeater.cc:503] Master 127.1.74.190:37805 was elected leader, sending a full tablet report...
I20260619 22:21:05.027179 4586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:05.030503 4601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:05.032253 4586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:05.032407 4586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "519ac758f35a480cb95fd226e92ba4e6"
format_stamp: "Formatted at 2026-06-19 22:21:05 on dist-test-slave-k84h"
server_key: "d6143883d86e61c1e3f8715127794be6"
server_key_iv: "2352a35a59e9d7e0e491e976b90fe45e"
server_key_version: "encryptionkey@0"
I20260619 22:21:05.032536 4586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:05.050124 4586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:05.053640 4586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:05.053861 4586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:05.054495 4586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:05.055414 4586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:05.055485 4586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:05.055558 4586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:05.055603 4586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:05.065701 4586 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:33769
I20260619 22:21:05.065735 4714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:33769 every 8 connection(s)
I20260619 22:21:05.066691 4586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:05.071225 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4586
I20260619 22:21:05.071369 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:05.071659 1322 external_mini_cluster.cc:1428] Setting key fc3e12a9f2444bebc9d25b7b0d5361cc
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907665, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:05.089660 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:05.068721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:49277 (local address 127.1.74.190:37805)
0619 22:21:05.068982 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:05.068987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:05.069964 (+ 977us) server_negotiation.cc:407] Connection header received
0619 22:21:05.071758 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:05.071762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:05.071826 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:05.071942 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:05.073559 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.074382 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.075686 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.075927 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.076048 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:05.085279 (+ 9231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:05.085283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:05.085285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:05.085317 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:05.087287 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:05.087963 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:05.087967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:05.087968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:05.088052 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:05.088441 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:05.088444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:05.088446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:05.088654 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:05.088765 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:05.089332 (+ 567us) server_negotiation.cc:299] Negotiation successful
0619 22:21:05.089472 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":59,"threads_started":1}
I20260619 22:21:05.090555 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:05.069297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37805 (local address 127.1.74.130:49277)
0619 22:21:05.069775 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:05.069819 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:05.071532 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:05.072136 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:05.072145 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:05.072645 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:05.073385 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:05.073405 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.074516 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.074519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:05.075558 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:05.075568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.076092 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.076096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:05.076368 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:05.080840 (+ 4472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:05.080860 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:05.084971 (+ 4111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:05.087483 (+ 2512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:05.087498 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:05.087514 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:05.087831 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:05.088149 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:05.088154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:05.088156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:05.088330 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:05.088766 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:05.088775 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:05.089067 (+ 292us) client_negotiation.cc:769] Sending connection context
0619 22:21:05.089269 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 22:21:05.089642 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":115,"threads_started":1}
I20260619 22:21:05.091897 4715 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37805
I20260619 22:21:05.092269 4715 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:05.092841 4715 heartbeater.cc:511] Master 127.1.74.190:37805 requested a full tablet report, sending...
I20260619 22:21:05.094120 4390 ts_manager.cc:194] Registered new tserver with Master: 519ac758f35a480cb95fd226e92ba4e6 (127.1.74.130:33769)
I20260619 22:21:05.094765 4390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:49277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:05.103034 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37805
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:05.218051 4721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:05.218411 4721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:05.218505 4721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:05.222960 4721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:05.223035 4721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:05.223117 4721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:05.228724 4721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:05.229866 4721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:05.231012 4721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:05.241132 4727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:05.241155 4721 server_base.cc:1034] running on GCE node
W20260619 22:21:05.241182 4726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:05.241204 4729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:05.241974 4721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:05.242709 4721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:05.243914 4721 hybrid_clock.cc:648] HybridClock initialized: now 1781907665243890 us; error 37 us; skew 500 ppm
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907665, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:05.246995 4721 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:05.248222 4721 webserver.cc:460] Webserver started at http://127.1.74.131:38503/ using document root <none> and password file <none>
I20260619 22:21:05.248982 4721 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:05.249091 4721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:05.249379 4721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:05.251863 4721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "598ba1db20974960980c64f65a940034"
format_stamp: "Formatted at 2026-06-19 22:21:05 on dist-test-slave-k84h"
server_key: "17ac05848037361b4cf0a52e624e5641"
server_key_iv: "a6688212b34ea1d9bf9238952c079e53"
server_key_version: "encryptionkey@0"
I20260619 22:21:05.252624 4721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "598ba1db20974960980c64f65a940034"
format_stamp: "Formatted at 2026-06-19 22:21:05 on dist-test-slave-k84h"
server_key: "17ac05848037361b4cf0a52e624e5641"
server_key_iv: "a6688212b34ea1d9bf9238952c079e53"
server_key_version: "encryptionkey@0"
I20260619 22:21:05.256747 4721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:05.259397 4736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:05.260707 4721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:05.260850 4721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "598ba1db20974960980c64f65a940034"
format_stamp: "Formatted at 2026-06-19 22:21:05 on dist-test-slave-k84h"
server_key: "17ac05848037361b4cf0a52e624e5641"
server_key_iv: "a6688212b34ea1d9bf9238952c079e53"
server_key_version: "encryptionkey@0"
I20260619 22:21:05.260979 4721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:05.275225 4721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:05.278363 4721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:05.278584 4721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:05.279223 4721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:05.280053 4721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:05.280124 4721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:05.280191 4721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:05.280225 4721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:05.289024 4721 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:44657
I20260619 22:21:05.289053 4849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:44657 every 8 connection(s)
I20260619 22:21:05.289921 4721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907665, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:05.299918 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4721
I20260619 22:21:05.300048 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:05.300361 1322 external_mini_cluster.cc:1428] Setting key 3d862faeaa1d1c3166da8f0448647c6b
I20260619 22:21:05.304037 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:05.291722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:43699 (local address 127.1.74.190:37805)
0619 22:21:05.291897 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:05.291902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:05.292726 (+ 824us) server_negotiation.cc:407] Connection header received
0619 22:21:05.294091 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:05.294094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:05.294144 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:05.294218 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:05.295514 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.296043 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.297302 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.297540 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.297619 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:05.300260 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:05.300263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:05.300265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:05.300289 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:05.302065 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:05.302626 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:05.302632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:05.302634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:05.302731 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:05.303083 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:05.303085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:05.303087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:05.303288 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:05.303377 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:05.303821 (+ 444us) server_negotiation.cc:299] Negotiation successful
0619 22:21:05.303923 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 22:21:05.304812 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:05.292129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37805 (local address 127.1.74.131:43699)
0619 22:21:05.292576 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:05.292619 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:05.293865 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:05.294402 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:05.294408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:05.294799 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:05.295372 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:05.295386 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.296187 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.296190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:05.297173 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:05.297183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.297690 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.297693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:05.297845 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:05.298500 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:05.298516 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:05.300094 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:05.302179 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:05.302184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:05.302195 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:05.302520 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:05.302820 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:05.302822 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:05.302823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:05.303003 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:05.303376 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:05.303381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:05.303609 (+ 228us) client_negotiation.cc:769] Sending connection context
0619 22:21:05.303754 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 22:21:05.303995 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
I20260619 22:21:05.305990 4850 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37805
I20260619 22:21:05.306197 4850 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:05.306623 4850 heartbeater.cc:511] Master 127.1.74.190:37805 requested a full tablet report, sending...
I20260619 22:21:05.307591 4390 ts_manager.cc:194] Registered new tserver with Master: 598ba1db20974960980c64f65a940034 (127.1.74.131:44657)
I20260619 22:21:05.308117 4390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:43699
I20260619 22:21:05.314445 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907665, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907665, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:05.339633 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:05.331192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.1.74.190:37805)
0619 22:21:05.331338 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:05.331343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:05.331501 (+ 158us) server_negotiation.cc:407] Connection header received
0619 22:21:05.331614 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:05.331616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:05.331660 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:05.331732 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:05.332422 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.332967 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.333784 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:05.334000 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:05.334077 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:05.336517 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:05.336520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:05.336522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:05.336548 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:05.338146 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:05.338522 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:05.338524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:05.338525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:05.338599 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:05.338814 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:05.338816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:05.338817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:05.339083 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:05.339187 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:05.339365 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 22:21:05.339453 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 22:21:05.340035 4390 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54402
I20260619 22:21:05.340914 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4452
I20260619 22:21:05.346601 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4586
I20260619 22:21:05.352367 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4721
I20260619 22:21:05.358088 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4358
2026-06-19T22:21:05Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3021 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4862](info): set up 2 sockets
Jun 19 22:21:05 dist-test-slave-k84h krb5kdc[4862](info): commencing operation
krb5kdc: starting...
W20260619 22:21:06.394950 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:06 dist-test-slave-k84h krb5kdc[4862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:06.521696 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39491
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:06.636500 4875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:06.636804 4875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:06.636847 4875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:06.640961 4875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:06.641034 4875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:06.641054 4875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:06.641072 4875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:06.641086 4875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:06.646504 4875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:06.647675 4875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:06.648669 4875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:06.657217 4880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:06.657217 4881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:06.657212 4883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:06.657572 4875 server_base.cc:1034] running on GCE node
I20260619 22:21:06.658208 4875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:06.659489 4875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:06.660688 4875 hybrid_clock.cc:648] HybridClock initialized: now 1781907666660674 us; error 29 us; skew 500 ppm
Jun 19 22:21:06 dist-test-slave-k84h krb5kdc[4862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907666, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:06.663635 4875 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:06.664534 4875 webserver.cc:460] Webserver started at http://127.1.74.190:38415/ using document root <none> and password file <none>
I20260619 22:21:06.665107 4875 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:06.665197 4875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:06.665433 4875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:06.667541 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "d5cc32255a6f44048eecfc5f5315d5e1"
format_stamp: "Formatted at 2026-06-19 22:21:06 on dist-test-slave-k84h"
server_key: "d523be1523c74adfa1f3d2d261d1c580"
server_key_iv: "36060d4cd850614427916116542ceab1"
server_key_version: "encryptionkey@0"
I20260619 22:21:06.668093 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "d5cc32255a6f44048eecfc5f5315d5e1"
format_stamp: "Formatted at 2026-06-19 22:21:06 on dist-test-slave-k84h"
server_key: "d523be1523c74adfa1f3d2d261d1c580"
server_key_iv: "36060d4cd850614427916116542ceab1"
server_key_version: "encryptionkey@0"
I20260619 22:21:06.671787 4875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:21:06.674453 4890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:06.675707 4875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:06.675840 4875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "d5cc32255a6f44048eecfc5f5315d5e1"
format_stamp: "Formatted at 2026-06-19 22:21:06 on dist-test-slave-k84h"
server_key: "d523be1523c74adfa1f3d2d261d1c580"
server_key_iv: "36060d4cd850614427916116542ceab1"
server_key_version: "encryptionkey@0"
I20260619 22:21:06.675959 4875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:06.696249 4875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:06.699707 4875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:06.699921 4875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:06.707477 4875 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39491
I20260619 22:21:06.707490 4942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39491 every 8 connection(s)
I20260619 22:21:06.708506 4875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:06.711591 4943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:06.717484 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1: Bootstrap starting.
I20260619 22:21:06.718509 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4875
I20260619 22:21:06.718647 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:06.719025 1322 external_mini_cluster.cc:1428] Setting key ff09943f09ed60f58bd9f8f84bfbefaa
I20260619 22:21:06.720871 4943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:06.721791 4943 log.cc:824] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:06.724228 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1: No bootstrap required, opened a new log
Jun 19 22:21:06 dist-test-slave-k84h krb5kdc[4862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:06.729107 4943 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39491 } }
I20260619 22:21:06.729457 4943 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:06.729543 4943 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cc32255a6f44048eecfc5f5315d5e1, State: Initialized, Role: FOLLOWER
I20260619 22:21:06.730144 4943 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39491 } }
I20260619 22:21:06.730335 4943 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:06.730414 4943 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:06.730530 4943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:06.731580 4943 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39491 } }
I20260619 22:21:06.731983 4943 leader_election.cc:302] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5cc32255a6f44048eecfc5f5315d5e1; no voters:
I20260619 22:21:06.732388 4943 leader_election.cc:288] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:06.732512 4948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:06.732749 4948 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [term 1 LEADER]: Becoming Leader. State: Replica: d5cc32255a6f44048eecfc5f5315d5e1, State: Running, Role: LEADER
I20260619 22:21:06.733081 4948 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39491 } }
I20260619 22:21:06.733839 4943 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:06.734393 4948 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5cc32255a6f44048eecfc5f5315d5e1. Latest consensus state: current_term: 1 leader_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39491 } } }
I20260619 22:21:06.734520 4948 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:06.734965 4949 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cc32255a6f44048eecfc5f5315d5e1" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39491 } } }
I20260619 22:21:06.735118 4949 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:06.735920 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:06.720343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.1.74.190:39491)
0619 22:21:06.720807 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:06.720823 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:06.721053 (+ 230us) server_negotiation.cc:407] Connection header received
0619 22:21:06.721864 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:06.721882 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:06.722335 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:06.722720 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:06.723793 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:06.724843 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:06.726051 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:06.726583 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:06.726704 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:06.729760 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:06.729767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:06.729786 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:06.729826 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:06.731897 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:06.732469 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:06.732477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:06.732483 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:06.732675 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:06.733023 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:06.733026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:06.733028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:06.733517 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:06.733722 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:06.734387 (+ 665us) server_negotiation.cc:299] Negotiation successful
0619 22:21:06.734738 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":136,"threads_started":1}
W20260619 22:21:06.739949 4963 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:21:06.740057 4963 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:21:06.740123 4953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:06.741086 4953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:06.747107 4953 catalog_manager.cc:1347] Generated new cluster ID: 907e576d32bd45ca9fc6874fd52cbf7e
I20260619 22:21:06.747197 4953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:06.764225 4953 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:06.765213 4953 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:06.772585 4953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5cc32255a6f44048eecfc5f5315d5e1: Generated new TSK 0
I20260619 22:21:06.773348 4953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:06.810060 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39491
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:06.927824 4969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:06.928121 4969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:06.928166 4969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:06.932379 4969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:06.932458 4969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:06.932554 4969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:06.938377 4969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:06.939559 4969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:06.940624 4969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:06.950763 4975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:06.950763 4974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:06.950763 4977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:06.951023 4969 server_base.cc:1034] running on GCE node
I20260619 22:21:06.951558 4969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:06.952288 4969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:06.953497 4969 hybrid_clock.cc:648] HybridClock initialized: now 1781907666953479 us; error 32 us; skew 500 ppm
Jun 19 22:21:06 dist-test-slave-k84h krb5kdc[4862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907666, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:06.956657 4969 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:06.957625 4969 webserver.cc:460] Webserver started at http://127.1.74.129:32939/ using document root <none> and password file <none>
I20260619 22:21:06.958223 4969 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:06.958320 4969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:06.958561 4969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:06.960780 4969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "8992a1fc83ec40d1a32ca9837899099b"
format_stamp: "Formatted at 2026-06-19 22:21:06 on dist-test-slave-k84h"
server_key: "befc90d80f65b7eee0b2ee77b3a1f59f"
server_key_iv: "14e0e18cc0f8b63f816f03193f8384e2"
server_key_version: "encryptionkey@0"
I20260619 22:21:06.961373 4969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "8992a1fc83ec40d1a32ca9837899099b"
format_stamp: "Formatted at 2026-06-19 22:21:06 on dist-test-slave-k84h"
server_key: "befc90d80f65b7eee0b2ee77b3a1f59f"
server_key_iv: "14e0e18cc0f8b63f816f03193f8384e2"
server_key_version: "encryptionkey@0"
I20260619 22:21:06.965405 4969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:06.968204 4984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:06.969650 4969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:06.969813 4969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "8992a1fc83ec40d1a32ca9837899099b"
format_stamp: "Formatted at 2026-06-19 22:21:06 on dist-test-slave-k84h"
server_key: "befc90d80f65b7eee0b2ee77b3a1f59f"
server_key_iv: "14e0e18cc0f8b63f816f03193f8384e2"
server_key_version: "encryptionkey@0"
I20260619 22:21:06.969952 4969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:06.992264 4969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:06.995811 4969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:06.996037 4969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:06.996665 4969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:06.997545 4969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:06.997618 4969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:06.997714 4969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:06.997764 4969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.006745 4969 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45027
I20260619 22:21:07.006774 5097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45027 every 8 connection(s)
I20260619 22:21:07.007707 4969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:07.017674 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 4969
I20260619 22:21:07.017836 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907666, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:07.018159 1322 external_mini_cluster.cc:1428] Setting key 94d6baf2254f9dc4ca98c45d998bdfb5
I20260619 22:21:07.023437 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.009719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:46569 (local address 127.1.74.190:39491)
0619 22:21:07.009907 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:07.009912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:07.010742 (+ 830us) server_negotiation.cc:407] Connection header received
0619 22:21:07.012234 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:07.012239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:07.012301 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:07.012393 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:07.013714 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.014312 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.015643 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.015898 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.015998 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.018864 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:07.018868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:07.018873 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:07.018905 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:07.020771 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.021346 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.021352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.021357 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.021468 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.021953 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.021956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.021957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.022155 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:07.022233 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:07.023008 (+ 775us) server_negotiation.cc:299] Negotiation successful
0619 22:21:07.023166 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 22:21:07.024247 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.010111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39491 (local address 127.1.74.129:46569)
0619 22:21:07.010576 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:07.010621 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:07.011976 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:07.012588 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:07.012596 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:07.012996 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:07.013528 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:07.013542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.014448 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.014451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:07.015502 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:07.015512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.016054 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.016057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:07.016211 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.016930 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:07.016947 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:07.018596 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:07.020916 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:07.020922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:07.020937 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:07.021241 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:07.021604 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:07.021610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:07.021613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:07.021841 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:07.022277 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:07.022286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:07.022667 (+ 381us) client_negotiation.cc:769] Sending connection context
0619 22:21:07.022973 (+ 306us) client_negotiation.cc:240] Negotiation successful
0619 22:21:07.023289 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":115,"threads_started":1}
I20260619 22:21:07.025573 5098 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39491
I20260619 22:21:07.025815 5098 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:07.026336 5098 heartbeater.cc:511] Master 127.1.74.190:39491 requested a full tablet report, sending...
I20260619 22:21:07.027931 4907 ts_manager.cc:194] Registered new tserver with Master: 8992a1fc83ec40d1a32ca9837899099b (127.1.74.129:45027)
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:07.029273 4907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:46569
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:21:07.046176 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39491
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:07.164367 5103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:07.164889 5103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:07.164999 5103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:07.170224 5103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:07.170315 5103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:07.170382 5103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:07.176215 5103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:07.177323 5103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:07.178336 5103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:07.188625 5109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:07.188635 5111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:07.188648 5108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:07.188709 5103 server_base.cc:1034] running on GCE node
I20260619 22:21:07.189314 5103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:07.190007 5103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:07.191231 5103 hybrid_clock.cc:648] HybridClock initialized: now 1781907667191210 us; error 37 us; skew 500 ppm
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:07.194142 5103 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:07.195099 5103 webserver.cc:460] Webserver started at http://127.1.74.130:33485/ using document root <none> and password file <none>
I20260619 22:21:07.195636 5103 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:07.195695 5103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:07.195878 5103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:07.197992 5103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "772a8c5ad4a6433e865cf14bba008abe"
format_stamp: "Formatted at 2026-06-19 22:21:07 on dist-test-slave-k84h"
server_key: "d9345bd27ccd9ea611e32dacddcd51af"
server_key_iv: "f8ab0e2190c46295d52ab36be950822c"
server_key_version: "encryptionkey@0"
I20260619 22:21:07.198531 5103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "772a8c5ad4a6433e865cf14bba008abe"
format_stamp: "Formatted at 2026-06-19 22:21:07 on dist-test-slave-k84h"
server_key: "d9345bd27ccd9ea611e32dacddcd51af"
server_key_iv: "f8ab0e2190c46295d52ab36be950822c"
server_key_version: "encryptionkey@0"
I20260619 22:21:07.202386 5103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 22:21:07.204969 5118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.206267 5103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:07.206447 5103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "772a8c5ad4a6433e865cf14bba008abe"
format_stamp: "Formatted at 2026-06-19 22:21:07 on dist-test-slave-k84h"
server_key: "d9345bd27ccd9ea611e32dacddcd51af"
server_key_iv: "f8ab0e2190c46295d52ab36be950822c"
server_key_version: "encryptionkey@0"
I20260619 22:21:07.206573 5103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:07.222661 5103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:07.226202 5103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:07.226419 5103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:07.227111 5103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:07.227977 5103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:07.228051 5103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.228127 5103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:07.228178 5103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.237767 5103 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:40517
I20260619 22:21:07.237784 5231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:40517 every 8 connection(s)
I20260619 22:21:07.238698 5103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:07.243121 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5103
I20260619 22:21:07.243250 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:07.243613 1322 external_mini_cluster.cc:1428] Setting key f31e71f856e7b48c3bc90786f7e77b85
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:21:07.253985 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.240622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:40587 (local address 127.1.74.190:39491)
0619 22:21:07.240776 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:07.240779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:07.241674 (+ 895us) server_negotiation.cc:407] Connection header received
0619 22:21:07.243282 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:07.243287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:07.243346 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:07.243416 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:07.244782 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.245438 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.247348 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.247576 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.247688 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.250402 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:07.250406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:07.250408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:07.250431 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:07.251919 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.252430 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.252433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.252435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.252513 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.252843 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.252846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.252848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.253029 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:07.253132 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:07.253669 (+ 537us) server_negotiation.cc:299] Negotiation successful
0619 22:21:07.253799 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 22:21:07.255103 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.241006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39491 (local address 127.1.74.130:40587)
0619 22:21:07.241497 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:07.241548 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:07.243000 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:07.243633 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:07.243643 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:07.244137 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:07.244645 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:07.244659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.245581 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.245585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:07.247217 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:07.247227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.247727 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.247731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:07.247927 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.248549 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:07.248566 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:07.250218 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:07.252032 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:07.252039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:07.252055 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:07.252326 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:07.252601 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:07.252605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:07.252607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:07.252743 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:07.253128 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:07.253135 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:07.253433 (+ 298us) client_negotiation.cc:769] Sending connection context
0619 22:21:07.253620 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 22:21:07.253954 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:21:07.256429 5232 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39491
I20260619 22:21:07.256682 5232 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:07.257179 5232 heartbeater.cc:511] Master 127.1.74.190:39491 requested a full tablet report, sending...
I20260619 22:21:07.258236 4907 ts_manager.cc:194] Registered new tserver with Master: 772a8c5ad4a6433e865cf14bba008abe (127.1.74.130:40517)
I20260619 22:21:07.258962 4907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:40587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:07.272224 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39491
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:39407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:07.383981 5237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:07.384225 5237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:07.384266 5237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:07.388396 5237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:07.388456 5237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:07.388541 5237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:07.393755 5237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:07.394799 5237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:07.395840 5237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:07.405037 5243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:07.405061 5245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:07.405071 5242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:07.405365 5237 server_base.cc:1034] running on GCE node
I20260619 22:21:07.405745 5237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:07.406364 5237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:07.407531 5237 hybrid_clock.cc:648] HybridClock initialized: now 1781907667407514 us; error 29 us; skew 500 ppm
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:07.410377 5237 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:07.411360 5237 webserver.cc:460] Webserver started at http://127.1.74.131:34181/ using document root <none> and password file <none>
I20260619 22:21:07.411906 5237 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:07.411968 5237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:07.412132 5237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:07.414127 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "7d65d1eb82a64baf8116185c517c0019"
format_stamp: "Formatted at 2026-06-19 22:21:07 on dist-test-slave-k84h"
server_key: "a51c982a2f8bcbfae766e84da184f3be"
server_key_iv: "b2c85b9aafe2b84b5ddc3dc094d09f12"
server_key_version: "encryptionkey@0"
I20260619 22:21:07.414640 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "7d65d1eb82a64baf8116185c517c0019"
format_stamp: "Formatted at 2026-06-19 22:21:07 on dist-test-slave-k84h"
server_key: "a51c982a2f8bcbfae766e84da184f3be"
server_key_iv: "b2c85b9aafe2b84b5ddc3dc094d09f12"
server_key_version: "encryptionkey@0"
I20260619 22:21:07.418386 5237 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:21:07.420925 5252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.422314 5237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:07.422480 5237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "7d65d1eb82a64baf8116185c517c0019"
format_stamp: "Formatted at 2026-06-19 22:21:07 on dist-test-slave-k84h"
server_key: "a51c982a2f8bcbfae766e84da184f3be"
server_key_iv: "b2c85b9aafe2b84b5ddc3dc094d09f12"
server_key_version: "encryptionkey@0"
I20260619 22:21:07.422595 5237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:07.433835 5237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:07.437085 5237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:07.437291 5237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:07.437884 5237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:07.438709 5237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:07.438777 5237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.438874 5237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:07.438918 5237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:07.447841 5237 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:35113
I20260619 22:21:07.447858 5365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:35113 every 8 connection(s)
I20260619 22:21:07.448971 5237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:07.458875 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5237
I20260619 22:21:07.459012 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:07.459282 1322 external_mini_cluster.cc:1428] Setting key 8f36b20005a1e1d0cd4cc2678baed994
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:07.464304 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.450967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49921 (local address 127.1.74.190:39491)
0619 22:21:07.451106 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:07.451111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:07.451947 (+ 836us) server_negotiation.cc:407] Connection header received
0619 22:21:07.453365 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:07.453368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:07.453424 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:07.453509 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:07.455043 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.455601 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.456854 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.457105 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.457204 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.460132 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:07.460136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:07.460139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:07.460172 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:07.461892 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.462460 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.462463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.462464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.462561 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.463259 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.463262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.463263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.463480 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:07.463560 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:07.464064 (+ 504us) server_negotiation.cc:299] Negotiation successful
0619 22:21:07.464175 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:21:07.465184 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.451343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39491 (local address 127.1.74.131:49921)
0619 22:21:07.451776 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:07.451822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:07.453117 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:07.453722 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:07.453730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:07.454124 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:07.454766 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:07.454787 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.455744 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.455747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:07.456693 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:07.456703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.457289 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.457293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:07.457448 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.458115 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:07.458132 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:07.459935 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:07.462031 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:07.462036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:07.462058 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:07.462331 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:07.462739 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:07.462742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:07.462743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:07.463165 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:07.463569 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:07.463575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:07.463830 (+ 255us) client_negotiation.cc:769] Sending connection context
0619 22:21:07.463994 (+ 164us) client_negotiation.cc:240] Negotiation successful
0619 22:21:07.464278 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260619 22:21:07.466497 5366 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39491
I20260619 22:21:07.466800 5366 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:07.467373 5366 heartbeater.cc:511] Master 127.1.74.190:39491 requested a full tablet report, sending...
I20260619 22:21:07.468576 4907 ts_manager.cc:194] Registered new tserver with Master: 7d65d1eb82a64baf8116185c517c0019 (127.1.74.131:35113)
I20260619 22:21:07.469481 4907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:49921
I20260619 22:21:07.473539 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:21:07.501070 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.491207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.1.74.129:45027)
0619 22:21:07.491472 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:07.491490 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:07.491517 (+ 27us) server_negotiation.cc:407] Connection header received
0619 22:21:07.491596 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:07.491600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:07.491804 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:07.491936 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:07.492629 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.493472 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.494345 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.494724 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.494823 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.497315 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:07.497326 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:07.497340 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:07.497366 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:07.499074 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.499522 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.499528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.499529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.499615 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.499940 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.499944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.499945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.500223 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:07.500414 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:07.500661 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 22:21:07.500842 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[4862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:07.513371 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:07.504132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54496 (local address 127.1.74.190:39491)
0619 22:21:07.504274 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:07.504280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:07.504393 (+ 113us) server_negotiation.cc:407] Connection header received
0619 22:21:07.504543 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:07.504546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:07.504605 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:07.504705 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:07.505358 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.505954 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.506798 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:07.507114 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:07.507230 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:07.509797 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:07.509801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:07.509804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:07.509840 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:07.511571 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.512015 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.512021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.512023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.512111 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:07.512394 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:07.512398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:07.512400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:07.512645 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:07.512794 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:07.513052 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 22:21:07.513182 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 22:21:07.513818 4907 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54496
I20260619 22:21:07.514498 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4969
I20260619 22:21:07.520403 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5103
I20260619 22:21:07.526749 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5237
I20260619 22:21:07.532512 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 4875
2026-06-19T22:21:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2174 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[5378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[5378](info): set up 2 sockets
Jun 19 22:21:07 dist-test-slave-k84h krb5kdc[5378](info): commencing operation
krb5kdc: starting...
W20260619 22:21:08.563107 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:08 dist-test-slave-k84h krb5kdc[5378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:08.687223 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40711
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:08.800364 5391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:08.800733 5391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:08.800823 5391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:08.805104 5391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:08.805238 5391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:08.805274 5391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:08.805317 5391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:08.805353 5391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:08.811039 5391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:08.812243 5391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:08.813328 5391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:08.821799 5396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:08.821802 5397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:08.821799 5399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:08.822052 5391 server_base.cc:1034] running on GCE node
I20260619 22:21:08.822739 5391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:08.824014 5391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:08.825217 5391 hybrid_clock.cc:648] HybridClock initialized: now 1781907668825203 us; error 40 us; skew 500 ppm
Jun 19 22:21:08 dist-test-slave-k84h krb5kdc[5378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907668, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:08.828356 5391 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:08.829108 5391 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:08.829176 5391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:08.829362 5391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:08.831624 5391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "1526708fcc5f47418efdd49caa2cc422"
format_stamp: "Formatted at 2026-06-19 22:21:08 on dist-test-slave-k84h"
server_key: "429ebbf3011a9d3f1dc215da576c5760"
server_key_iv: "4753aa591b79e62ee08e7fb8a291bba2"
server_key_version: "encryptionkey@0"
I20260619 22:21:08.832233 5391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "1526708fcc5f47418efdd49caa2cc422"
format_stamp: "Formatted at 2026-06-19 22:21:08 on dist-test-slave-k84h"
server_key: "429ebbf3011a9d3f1dc215da576c5760"
server_key_iv: "4753aa591b79e62ee08e7fb8a291bba2"
server_key_version: "encryptionkey@0"
I20260619 22:21:08.836369 5391 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:08.839075 5405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:08.840371 5391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:08.840524 5391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "1526708fcc5f47418efdd49caa2cc422"
format_stamp: "Formatted at 2026-06-19 22:21:08 on dist-test-slave-k84h"
server_key: "429ebbf3011a9d3f1dc215da576c5760"
server_key_iv: "4753aa591b79e62ee08e7fb8a291bba2"
server_key_version: "encryptionkey@0"
I20260619 22:21:08.840639 5391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:08.859433 5391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:08.862773 5391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:08.863020 5391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:08.870390 5391 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40711
I20260619 22:21:08.870664 5457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40711 every 8 connection(s)
I20260619 22:21:08.871111 5391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:08.873966 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5391
I20260619 22:21:08.874047 5458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:08.874125 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:08.874410 1322 external_mini_cluster.cc:1428] Setting key 68b491d92b30b71537e83ff07d467d4a
I20260619 22:21:08.880985 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422: Bootstrap starting.
Jun 19 22:21:08 dist-test-slave-k84h krb5kdc[5378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:08.883868 5458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:08.884739 5458 log.cc:824] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:08.887409 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422: No bootstrap required, opened a new log
I20260619 22:21:08.890120 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:08.875872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.1.74.190:40711)
0619 22:21:08.876344 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:08.876360 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:08.876396 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:21:08.877237 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:08.877258 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:08.877599 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:08.877882 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:08.878833 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:08.879872 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:08.881016 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:08.881571 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:08.881707 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:08.884531 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:08.884538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:08.884556 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:08.884595 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:08.886716 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:08.887250 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:08.887256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:08.887262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:08.887412 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:08.887744 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:08.887748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:08.887753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:08.888284 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:08.888498 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:08.888709 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 22:21:08.889021 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":160,"threads_started":1}
I20260619 22:21:08.891851 5458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1526708fcc5f47418efdd49caa2cc422" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40711 } }
I20260619 22:21:08.892096 5458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:08.892163 5458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1526708fcc5f47418efdd49caa2cc422, State: Initialized, Role: FOLLOWER
I20260619 22:21:08.892637 5458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1526708fcc5f47418efdd49caa2cc422" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40711 } }
I20260619 22:21:08.892766 5458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:08.892838 5458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:08.892944 5458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:08.893936 5458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1526708fcc5f47418efdd49caa2cc422" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40711 } }
I20260619 22:21:08.894286 5458 leader_election.cc:302] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1526708fcc5f47418efdd49caa2cc422; no voters:
I20260619 22:21:08.894620 5458 leader_election.cc:288] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:08.894737 5463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:08.894994 5463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [term 1 LEADER]: Becoming Leader. State: Replica: 1526708fcc5f47418efdd49caa2cc422, State: Running, Role: LEADER
I20260619 22:21:08.895318 5463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1526708fcc5f47418efdd49caa2cc422" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40711 } }
I20260619 22:21:08.895942 5458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:08.897125 5465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1526708fcc5f47418efdd49caa2cc422. Latest consensus state: current_term: 1 leader_uuid: "1526708fcc5f47418efdd49caa2cc422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1526708fcc5f47418efdd49caa2cc422" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40711 } } }
I20260619 22:21:08.897136 5464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1526708fcc5f47418efdd49caa2cc422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1526708fcc5f47418efdd49caa2cc422" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40711 } } }
I20260619 22:21:08.897282 5465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:08.897285 5464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:08.897713 5472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:08.901072 5472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:08.907261 5472 catalog_manager.cc:1347] Generated new cluster ID: 4981b414d459402d8a0cb63688c273fc
I20260619 22:21:08.907346 5472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:08.920987 5472 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:08.921983 5472 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:08.929246 5472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422: Generated new TSK 0
I20260619 22:21:08.929989 5472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:08.963855 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40711
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:09.075644 5484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:09.075927 5484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:09.075974 5484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:09.080129 5484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:09.080202 5484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:09.080266 5484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:09.085770 5484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:09.086977 5484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:09.088039 5484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:09.097684 5490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:09.097684 5489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:09.097684 5492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:09.097939 5484 server_base.cc:1034] running on GCE node
I20260619 22:21:09.098438 5484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:09.099190 5484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:09.100370 5484 hybrid_clock.cc:648] HybridClock initialized: now 1781907669100347 us; error 37 us; skew 500 ppm
Jun 19 22:21:09 dist-test-slave-k84h krb5kdc[5378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:09.103539 5484 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:09.104333 5484 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:09.104442 5484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:09.104681 5484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:09.106828 5484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "5878d39f9e174db9868d90e9d7825c7d"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "fe0bba9e28c657e3032092039d7d6c9b"
server_key_iv: "d722674f7bcdb60d4fc41f86a67482f0"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.107519 5484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "5878d39f9e174db9868d90e9d7825c7d"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "fe0bba9e28c657e3032092039d7d6c9b"
server_key_iv: "d722674f7bcdb60d4fc41f86a67482f0"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.111800 5484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260619 22:21:09.114569 5498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.116351 5484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 22:21:09.116536 5484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "5878d39f9e174db9868d90e9d7825c7d"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "fe0bba9e28c657e3032092039d7d6c9b"
server_key_iv: "d722674f7bcdb60d4fc41f86a67482f0"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.116673 5484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:09.153446 5484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:09.156875 5484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:09.157090 5484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:09.157738 5484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:09.158597 5484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:09.158672 5484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.158744 5484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:09.158783 5484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.167691 5484 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:40865
I20260619 22:21:09.167714 5611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:40865 every 8 connection(s)
I20260619 22:21:09.168318 5484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:09.170693 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5484
I20260619 22:21:09.170817 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:09.171183 1322 external_mini_cluster.cc:1428] Setting key d42190b402ec7dc9290ab829b75746b1
Jun 19 22:21:09 dist-test-slave-k84h krb5kdc[5378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:21:09.184260 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.170219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:51147 (local address 127.1.74.190:40711)
0619 22:21:09.170378 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.170383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.171478 (+ 1095us) server_negotiation.cc:407] Connection header received
0619 22:21:09.173090 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.173093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.173135 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.173202 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:09.174916 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.175790 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.176960 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.177400 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.177532 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.180292 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:09.180298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:09.180305 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:09.180341 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:09.181893 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.182462 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.182465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.182466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.182546 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.183010 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.183012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.183014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.183199 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:09.183277 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.183935 (+ 658us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.184055 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:21:09.185070 5614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.170694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40711 (local address 127.1.74.129:51147)
0619 22:21:09.171295 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.171356 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.172863 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.173404 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.173416 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.173976 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:09.174714 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.174730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.175915 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.175917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.176804 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.176813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.177559 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.177562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.177704 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.178355 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:09.178371 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:09.180103 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:09.182011 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:09.182016 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:09.182040 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:09.182378 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:09.182663 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:09.182666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:09.182668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:09.182826 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:09.183300 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:09.183306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:09.183582 (+ 276us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.183835 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.184193 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":167,"threads_started":1}
I20260619 22:21:09.186404 5612 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40711
I20260619 22:21:09.186663 5612 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:09.187196 5612 heartbeater.cc:511] Master 127.1.74.190:40711 requested a full tablet report, sending...
I20260619 22:21:09.188730 5419 ts_manager.cc:194] Registered new tserver with Master: 5878d39f9e174db9868d90e9d7825c7d (127.1.74.129:40865)
I20260619 22:21:09.190011 5419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:51147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:21:09.199955 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40711
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:09.311970 5617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:09.312266 5617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:09.312341 5617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:09.316324 5617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:09.316411 5617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:09.316505 5617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:09.321874 5617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:09.323042 5617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:09.324088 5617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:09.333681 5622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:09.333679 5623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:09.333679 5625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:09.334075 5617 server_base.cc:1034] running on GCE node
I20260619 22:21:09.334545 5617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:09.335312 5617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:09.336526 5617 hybrid_clock.cc:648] HybridClock initialized: now 1781907669336510 us; error 36 us; skew 500 ppm
Jun 19 22:21:09 dist-test-slave-k84h krb5kdc[5378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:09.339480 5617 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:09.340231 5617 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:09.340322 5617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:09.340546 5617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:09.342664 5617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "87783e0f755d41df8017fce886332d75"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "cea509d1bf40557845f7dab3d2fc898f"
server_key_iv: "2ca90cb3738bbe1632ffc60b8214827e"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.343324 5617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "87783e0f755d41df8017fce886332d75"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "cea509d1bf40557845f7dab3d2fc898f"
server_key_iv: "2ca90cb3738bbe1632ffc60b8214827e"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.347460 5617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:21:09.350059 5631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.351487 5617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:09.351624 5617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "87783e0f755d41df8017fce886332d75"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "cea509d1bf40557845f7dab3d2fc898f"
server_key_iv: "2ca90cb3738bbe1632ffc60b8214827e"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.351744 5617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:09.392797 5617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:09.396394 5617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:09.396606 5617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:09.397276 5617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:09.398228 5617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:09.398306 5617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.398387 5617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:09.398437 5617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.407341 5617 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:34391
I20260619 22:21:09.407361 5744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:34391 every 8 connection(s)
I20260619 22:21:09.407950 5617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:09.416970 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5617
Jun 19 22:21:09 dist-test-slave-k84h krb5kdc[5378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:09.417091 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:09.417393 1322 external_mini_cluster.cc:1428] Setting key e48f23fb956a7f526fddf099f8d6a3a5
I20260619 22:21:09.421955 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.409655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:39815 (local address 127.1.74.190:40711)
0619 22:21:09.409798 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.409801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.410650 (+ 849us) server_negotiation.cc:407] Connection header received
0619 22:21:09.412020 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.412023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.412071 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.412148 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:09.413304 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.413830 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.414990 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.415217 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.415295 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.417813 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:09.417817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:09.417819 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:09.417847 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:09.419485 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.420115 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.420118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.420119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.420200 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.420650 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.420652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.420653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.420938 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:09.421045 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.421588 (+ 543us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.421740 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 22:21:09.422725 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.410020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40711 (local address 127.1.74.130:39815)
0619 22:21:09.410493 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.410538 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.411808 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.412320 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.412327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.412698 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:09.413174 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.413187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.413942 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.413944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.414835 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.414876 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.415339 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.415341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.415491 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.416024 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:09.416040 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:09.417624 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:09.419598 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:09.419606 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:09.419627 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:09.420009 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:09.420287 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:09.420293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:09.420298 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:09.420539 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:09.421045 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:09.421051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:09.421299 (+ 248us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.421522 (+ 223us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.421804 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
I20260619 22:21:09.423933 5745 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40711
I20260619 22:21:09.424237 5745 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:09.424827 5745 heartbeater.cc:511] Master 127.1.74.190:40711 requested a full tablet report, sending...
I20260619 22:21:09.426103 5419 ts_manager.cc:194] Registered new tserver with Master: 87783e0f755d41df8017fce886332d75 (127.1.74.130:34391)
I20260619 22:21:09.426995 5419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:39815
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:09.445551 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40711
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:09.555655 5750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:09.555920 5750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:09.555969 5750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:09.560011 5750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:09.560083 5750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:09.560145 5750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:09.565419 5750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:09.566455 5750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:09.567449 5750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:09.576799 5755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:09.576916 5750 server_base.cc:1034] running on GCE node
W20260619 22:21:09.576800 5756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:09.576799 5758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:09.577447 5750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:09.578087 5750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:09.579246 5750 hybrid_clock.cc:648] HybridClock initialized: now 1781907669579230 us; error 39 us; skew 500 ppm
Jun 19 22:21:09 dist-test-slave-k84h krb5kdc[5378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:09.582129 5750 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:09.583024 5750 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:09.583117 5750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:09.583350 5750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:09.585484 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "77e94342891f443781cbc94af8c0ee87"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "5453c784774f7f6959c9709593bd4365"
server_key_iv: "be7b3a1e76068f6b77a8a67f13de4942"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.586098 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "77e94342891f443781cbc94af8c0ee87"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "5453c784774f7f6959c9709593bd4365"
server_key_iv: "be7b3a1e76068f6b77a8a67f13de4942"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.590058 5750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:21:09.592712 5764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.593948 5750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:09.594079 5750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "77e94342891f443781cbc94af8c0ee87"
format_stamp: "Formatted at 2026-06-19 22:21:09 on dist-test-slave-k84h"
server_key: "5453c784774f7f6959c9709593bd4365"
server_key_iv: "be7b3a1e76068f6b77a8a67f13de4942"
server_key_version: "encryptionkey@0"
I20260619 22:21:09.594197 5750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:09.620347 5750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:09.623637 5750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:09.623840 5750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:09.624440 5750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:09.625275 5750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:09.625342 5750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.625412 5750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:09.625458 5750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:09.634200 5750 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:42881
I20260619 22:21:09.634217 5877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:42881 every 8 connection(s)
I20260619 22:21:09.634822 5750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:09.642198 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5750
I20260619 22:21:09.642333 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:09.642654 1322 external_mini_cluster.cc:1428] Setting key 7e79edae5d65554373e35abfb997694f
Jun 19 22:21:09 dist-test-slave-k84h krb5kdc[5378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907669, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:09.648567 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.636566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:35663 (local address 127.1.74.190:40711)
0619 22:21:09.636719 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.636724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.637566 (+ 842us) server_negotiation.cc:407] Connection header received
0619 22:21:09.638888 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.638894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.638951 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.639031 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:09.640187 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.640714 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.641818 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.642050 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.642131 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.645221 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:09.645224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:09.645226 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:09.645249 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:09.646699 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.647201 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.647204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.647205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.647277 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.647559 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.647561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.647562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.647728 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:09.647806 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.648311 (+ 505us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.648439 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:21:09.649372 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.636993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40711 (local address 127.1.74.131:35663)
0619 22:21:09.637423 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.637465 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.638654 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.639191 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.639198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.639577 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:09.640049 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.640062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.640824 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.640826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.641715 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.641723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.642205 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.642210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.642444 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.643118 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:09.643133 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:09.645084 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:09.646799 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:09.646803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:09.646816 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:09.647116 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:09.647353 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:09.647355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:09.647357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:09.647485 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:09.647812 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:09.647817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:09.648057 (+ 240us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.648221 (+ 164us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.648472 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260619 22:21:09.650456 5878 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40711
I20260619 22:21:09.650693 5878 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:09.651172 5878 heartbeater.cc:511] Master 127.1.74.190:40711 requested a full tablet report, sending...
I20260619 22:21:09.652119 5419 ts_manager.cc:194] Registered new tserver with Master: 77e94342891f443781cbc94af8c0ee87 (127.1.74.131:42881)
I20260619 22:21:09.652670 5419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:35663
I20260619 22:21:09.656374 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:21:09.665575 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.658229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.1.74.190:40711)
0619 22:21:09.658360 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.658367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.658530 (+ 163us) server_negotiation.cc:407] Connection header received
0619 22:21:09.658676 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.658679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.658725 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.658801 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:09.659511 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.660035 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.660870 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.661103 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.661203 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.662500 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:09.662504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:09.662507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:09.662530 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:09.663958 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.664363 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.664367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.664369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.664448 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:09.664710 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:09.664713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:09.664715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:09.664935 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:09.665069 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.665297 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.665412 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 22:21:09.668299 5419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:09.671417 5419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:09.685416 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.680396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.1.74.131:42881)
0619 22:21:09.680721 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.680732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.681032 (+ 300us) server_negotiation.cc:407] Connection header received
0619 22:21:09.681115 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.681120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.681333 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.681475 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:09.682007 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.682719 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.684168 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.684787 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.684908 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.685045 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.685173 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.685261 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":153,"threads_started":1}
I20260619 22:21:09.685442 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.679672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:42881 (local address 127.0.0.1:44910)
0619 22:21:09.680842 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.680854 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.680937 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.681645 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.681649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.681678 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:09.681858 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.681863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.682875 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.682880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.684053 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.684062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.684904 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.684906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.684992 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.685021 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.685137 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.685251 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"mutex_wait_us":63,"spinlock_wait_cycles":6016,"thread_start_us":79,"threads_started":1}
I20260619 22:21:09.686388 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.679606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34391 (local address 127.0.0.1:45402)
0619 22:21:09.680462 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.680513 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.680630 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.681478 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.681484 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.681505 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:09.681639 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.681646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.683173 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.683179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.684523 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.684533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.686035 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.686038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.686124 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.686145 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.686204 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.686264 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":71,"thread_start_us":88,"threads_started":1}
I20260619 22:21:09.686705 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.680241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.1.74.129:40865)
0619 22:21:09.680822 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.680826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.681074 (+ 248us) server_negotiation.cc:407] Connection header received
0619 22:21:09.681208 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.681212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.681353 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.681511 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:09.681973 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.683112 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.684649 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.685421 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.685518 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.686029 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.686458 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.686549 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":93,"threads_started":1}
I20260619 22:21:09.687007 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.679933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40865 (local address 127.0.0.1:53418)
0619 22:21:09.680973 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.680986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.681099 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.681645 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.681649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.681680 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:09.681855 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.681862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.683245 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.683248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.684523 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.684533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.686092 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.686096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.686181 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.686202 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.686273 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.686364 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":91,"spinlock_wait_cycles":3456,"thread_start_us":85,"threads_started":1}
I20260619 22:21:09.687022 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.679731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.1.74.130:34391)
0619 22:21:09.681031 (+ 1300us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.681043 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.681062 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:21:09.681122 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.681126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.681321 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.681475 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:09.681761 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.683022 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.685115 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.685905 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.686006 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.686123 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.686748 (+ 625us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.686884 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192,"thread_start_us":140,"threads_started":1}
I20260619 22:21:09.688730 5812 tablet_service.cc:1467] Processing CreateTablet for tablet 80ba9bca9c754e7fbec22c7bd1bd7031 (DEFAULT_TABLE table=test-table [id=a089bf70a88e40de89725448bd713634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:09.689204 5679 tablet_service.cc:1467] Processing CreateTablet for tablet 80ba9bca9c754e7fbec22c7bd1bd7031 (DEFAULT_TABLE table=test-table [id=a089bf70a88e40de89725448bd713634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:09.689873 5812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80ba9bca9c754e7fbec22c7bd1bd7031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:09.689990 5679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80ba9bca9c754e7fbec22c7bd1bd7031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:09.689599 5546 tablet_service.cc:1467] Processing CreateTablet for tablet 80ba9bca9c754e7fbec22c7bd1bd7031 (DEFAULT_TABLE table=test-table [id=a089bf70a88e40de89725448bd713634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:09.690539 5546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80ba9bca9c754e7fbec22c7bd1bd7031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:09.696128 5895 tablet_bootstrap.cc:492] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Bootstrap starting.
I20260619 22:21:09.697505 5896 tablet_bootstrap.cc:492] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: Bootstrap starting.
I20260619 22:21:09.699113 5897 tablet_bootstrap.cc:492] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Bootstrap starting.
I20260619 22:21:09.699260 5895 tablet_bootstrap.cc:654] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:09.699648 5896 tablet_bootstrap.cc:654] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:09.700147 5895 log.cc:824] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:09.700404 5896 log.cc:824] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:09.700942 5897 tablet_bootstrap.cc:654] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:09.702091 5895 tablet_bootstrap.cc:492] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: No bootstrap required, opened a new log
I20260619 22:21:09.702186 5897 log.cc:824] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:09.702322 5895 ts_tablet_manager.cc:1397] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:21:09.702757 5896 tablet_bootstrap.cc:492] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: No bootstrap required, opened a new log
I20260619 22:21:09.702967 5896 ts_tablet_manager.cc:1397] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:21:09.704080 5897 tablet_bootstrap.cc:492] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: No bootstrap required, opened a new log
I20260619 22:21:09.704252 5897 ts_tablet_manager.cc:1397] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 22:21:09.706167 5896 raft_consensus.cc:348] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.706408 5896 raft_consensus.cc:374] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:09.706190 5895 raft_consensus.cc:348] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.706468 5896 raft_consensus.cc:729] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5878d39f9e174db9868d90e9d7825c7d, State: Initialized, Role: FOLLOWER
I20260619 22:21:09.706473 5895 raft_consensus.cc:374] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:09.706524 5895 raft_consensus.cc:729] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87783e0f755d41df8017fce886332d75, State: Initialized, Role: FOLLOWER
I20260619 22:21:09.706980 5896 consensus_queue.cc:260] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.706988 5895 consensus_queue.cc:260] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.707878 5745 heartbeater.cc:503] Master 127.1.74.190:40711 was elected leader, sending a full tablet report...
I20260619 22:21:09.707999 5895 ts_tablet_manager.cc:1428] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:21:09.707959 5897 raft_consensus.cc:348] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.708207 5897 raft_consensus.cc:374] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:09.708271 5897 raft_consensus.cc:729] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e94342891f443781cbc94af8c0ee87, State: Initialized, Role: FOLLOWER
I20260619 22:21:09.708669 5897 consensus_queue.cc:260] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.708983 5896 ts_tablet_manager.cc:1428] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:21:09.709118 5612 heartbeater.cc:503] Master 127.1.74.190:40711 was elected leader, sending a full tablet report...
I20260619 22:21:09.709478 5878 heartbeater.cc:503] Master 127.1.74.190:40711 was elected leader, sending a full tablet report...
I20260619 22:21:09.709523 5897 ts_tablet_manager.cc:1428] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:21:09.753500 5901 raft_consensus.cc:482] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:09.753738 5901 raft_consensus.cc:504] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.755198 5901 leader_election.cc:288] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87783e0f755d41df8017fce886332d75 (127.1.74.130:34391), 77e94342891f443781cbc94af8c0ee87 (127.1.74.131:42881)
I20260619 22:21:09.759423 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.755590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:35965 (local address 127.1.74.130:34391)
0619 22:21:09.755721 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.755726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.756104 (+ 378us) server_negotiation.cc:407] Connection header received
0619 22:21:09.756304 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.756307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.756362 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.756460 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:09.756765 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.757271 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.758436 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.758834 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.758981 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.759032 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.759229 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.759288 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 22:21:09.759452 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.755399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:34391 (local address 127.1.74.129:35965)
0619 22:21:09.756018 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.756033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.756140 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.756471 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.756474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.756488 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:09.756652 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.756658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.757378 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.757383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.758312 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.758321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.758981 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.758985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.759114 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.759132 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.759215 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.759286 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":300,"spinlock_wait_cycles":4608,"thread_start_us":117,"threads_started":1}
I20260619 22:21:09.759579 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.755578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:39509 (local address 127.1.74.131:42881)
0619 22:21:09.755719 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.755725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.756013 (+ 288us) server_negotiation.cc:407] Connection header received
0619 22:21:09.756203 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.756207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.756272 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.756347 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:09.756839 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.757361 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.758495 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.758930 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.759068 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.759150 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.759379 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.759448 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 22:21:09.759604 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.755391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:42881 (local address 127.1.74.129:39509)
0619 22:21:09.755895 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.755912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:09.756045 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:09.756492 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:09.756494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:09.756532 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:09.756684 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.756689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.757486 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.757493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.758370 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:09.758380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.759068 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.759073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:09.759243 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.759263 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 22:21:09.759365 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 22:21:09.759438 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":77,"spinlock_wait_cycles":28800,"thread_start_us":123,"threads_started":1}
I20260619 22:21:09.760300 5832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80ba9bca9c754e7fbec22c7bd1bd7031" candidate_uuid: "5878d39f9e174db9868d90e9d7825c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e94342891f443781cbc94af8c0ee87" is_pre_election: true
I20260619 22:21:09.760311 5699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80ba9bca9c754e7fbec22c7bd1bd7031" candidate_uuid: "5878d39f9e174db9868d90e9d7825c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87783e0f755d41df8017fce886332d75" is_pre_election: true
I20260619 22:21:09.760591 5832 raft_consensus.cc:2454] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5878d39f9e174db9868d90e9d7825c7d in term 0.
I20260619 22:21:09.760605 5699 raft_consensus.cc:2454] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5878d39f9e174db9868d90e9d7825c7d in term 0.
I20260619 22:21:09.761055 5500 leader_election.cc:302] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878d39f9e174db9868d90e9d7825c7d, 77e94342891f443781cbc94af8c0ee87; no voters:
I20260619 22:21:09.761333 5901 raft_consensus.cc:2781] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:09.761425 5901 raft_consensus.cc:482] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:09.761469 5901 raft_consensus.cc:3037] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:09.762463 5901 raft_consensus.cc:504] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.762766 5901 leader_election.cc:288] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [CANDIDATE]: Term 1 election: Requested vote from peers 87783e0f755d41df8017fce886332d75 (127.1.74.130:34391), 77e94342891f443781cbc94af8c0ee87 (127.1.74.131:42881)
I20260619 22:21:09.763114 5832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80ba9bca9c754e7fbec22c7bd1bd7031" candidate_uuid: "5878d39f9e174db9868d90e9d7825c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e94342891f443781cbc94af8c0ee87"
I20260619 22:21:09.763113 5699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80ba9bca9c754e7fbec22c7bd1bd7031" candidate_uuid: "5878d39f9e174db9868d90e9d7825c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87783e0f755d41df8017fce886332d75"
I20260619 22:21:09.763217 5832 raft_consensus.cc:3037] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:09.763257 5699 raft_consensus.cc:3037] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:09.764232 5832 raft_consensus.cc:2454] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5878d39f9e174db9868d90e9d7825c7d in term 1.
I20260619 22:21:09.764424 5699 raft_consensus.cc:2454] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5878d39f9e174db9868d90e9d7825c7d in term 1.
I20260619 22:21:09.764540 5500 leader_election.cc:302] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5878d39f9e174db9868d90e9d7825c7d, 77e94342891f443781cbc94af8c0ee87; no voters:
I20260619 22:21:09.764705 5901 raft_consensus.cc:2781] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:09.764921 5901 raft_consensus.cc:686] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 1 LEADER]: Becoming Leader. State: Replica: 5878d39f9e174db9868d90e9d7825c7d, State: Running, Role: LEADER
I20260619 22:21:09.765245 5901 consensus_queue.cc:237] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } }
I20260619 22:21:09.768227 5419 catalog_manager.cc:5495] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5878d39f9e174db9868d90e9d7825c7d (127.1.74.129). New cstate: current_term: 1 leader_uuid: "5878d39f9e174db9868d90e9d7825c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5878d39f9e174db9868d90e9d7825c7d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40865 } health_report { overall_health: HEALTHY } } }
I20260619 22:21:09.783090 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.779709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.1.74.129:40865)
0619 22:21:09.779822 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:09.779832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:09.779845 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:21:09.779926 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:09.779929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:09.779980 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:09.780068 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:09.780265 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.780772 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.781578 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:09.781813 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:09.781904 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:09.782235 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:09.782672 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:09.782766 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:09.782914 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 22:21:09.782978 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 22:21:09.789630 5832 raft_consensus.cc:1261] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 1 FOLLOWER]: Refusing update from remote peer 5878d39f9e174db9868d90e9d7825c7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:09.790408 5699 raft_consensus.cc:1261] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 1 FOLLOWER]: Refusing update from remote peer 5878d39f9e174db9868d90e9d7825c7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:09.790386 5906 consensus_queue.cc:1035] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e94342891f443781cbc94af8c0ee87" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:09.791010 5906 consensus_queue.cc:1035] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "87783e0f755d41df8017fce886332d75" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 34391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:09.798380 5910 mvcc.cc:205] Tried to move back new op lower bound from 7298693815450775552 to 7298693815360913408. Current Snapshot: MvccSnapshot[applied={T|T < 7298693815450775552}]
I20260619 22:21:09.807236 5419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:09.807476 5419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:09.810268 5419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1526708fcc5f47418efdd49caa2cc422: Sending DeleteTablet for 3 replicas of tablet 80ba9bca9c754e7fbec22c7bd1bd7031
I20260619 22:21:09.811251 5546 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ba9bca9c754e7fbec22c7bd1bd7031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:09 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53418
I20260619 22:21:09.811225 5812 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ba9bca9c754e7fbec22c7bd1bd7031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:09 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:44910
I20260619 22:21:09.811226 5679 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ba9bca9c754e7fbec22c7bd1bd7031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:09 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:45402
I20260619 22:21:09.811779 5919 tablet_replica.cc:330] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d: stopping tablet replica
I20260619 22:21:09.811777 5920 tablet_replica.cc:330] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: stopping tablet replica
I20260619 22:21:09.812120 5920 raft_consensus.cc:2229] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:09.812120 5919 raft_consensus.cc:2229] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 5878d39f9e174db9868d90e9d7825c7d [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:09.812453 5920 raft_consensus.cc:2258] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:09.812737 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5484
I20260619 22:21:09.813959 5921 tablet_replica.cc:330] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: stopping tablet replica
I20260619 22:21:09.814293 5921 raft_consensus.cc:2229] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:09.814373 5920 ts_tablet_manager.cc:1905] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:09.814596 5921 raft_consensus.cc:2258] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:09.816394 5921 ts_tablet_manager.cc:1905] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:09.817150 5920 ts_tablet_manager.cc:1918] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:09.817248 5920 log.cc:1196] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/80ba9bca9c754e7fbec22c7bd1bd7031
I20260619 22:21:09.817574 5920 ts_tablet_manager.cc:1939] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 77e94342891f443781cbc94af8c0ee87: Deleting consensus metadata
I20260619 22:21:09.819005 5407 catalog_manager.cc:4841] TS 77e94342891f443781cbc94af8c0ee87 (127.1.74.131:42881): tablet 80ba9bca9c754e7fbec22c7bd1bd7031 (table test-table [id=a089bf70a88e40de89725448bd713634]) successfully deleted
I20260619 22:21:09.819164 5921 ts_tablet_manager.cc:1918] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:09.819257 5921 log.cc:1196] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/80ba9bca9c754e7fbec22c7bd1bd7031
I20260619 22:21:09.819600 5921 ts_tablet_manager.cc:1939] T 80ba9bca9c754e7fbec22c7bd1bd7031 P 87783e0f755d41df8017fce886332d75: Deleting consensus metadata
W20260619 22:21:09.820201 5409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:40865 (error 108)
I20260619 22:21:09.820827 5406 catalog_manager.cc:4841] TS 87783e0f755d41df8017fce886332d75 (127.1.74.130:34391): tablet 80ba9bca9c754e7fbec22c7bd1bd7031 (table test-table [id=a089bf70a88e40de89725448bd713634]) successfully deleted
I20260619 22:21:09.821056 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5617
I20260619 22:21:09.821242 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:09.820935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40865 (local address 127.0.0.1:53440)
0619 22:21:09.821062 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:09.821140 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:40865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 22:21:09.821982 5409 catalog_manager.cc:4584] TS 5878d39f9e174db9868d90e9d7825c7d (127.1.74.129:40865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 80ba9bca9c754e7fbec22c7bd1bd7031: Network error: Client connection negotiation failed: client connection to 127.1.74.129:40865: connect: Connection refused (error 111)
I20260619 22:21:09.827100 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5750
I20260619 22:21:09.832669 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5391
2026-06-19T22:21:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2300 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T22:21:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:09Z Disabled control of system clock
I20260619 22:21:09.863849 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39469
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 22:21:09.976078 5926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:09.976393 5926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:09.976442 5926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:09.983664 5926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:09.983752 5926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:09.983769 5926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:09.983788 5926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:09.983803 5926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:09.989214 5926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:09.990351 5926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:09.991767 5926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:10.000449 5932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.000449 5934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:10.000910 5926 server_base.cc:1034] running on GCE node
W20260619 22:21:10.000485 5931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:10.001891 5926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:10.003175 5926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:10.004356 5926 hybrid_clock.cc:648] HybridClock initialized: now 1781907670004315 us; error 60 us; skew 500 ppm
I20260619 22:21:10.006351 5926 webserver.cc:460] Webserver started at http://127.1.74.190:37725/ using document root <none> and password file <none>
I20260619 22:21:10.007079 5926 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:10.007174 5926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:10.008159 5926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:10.010705 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "482124398b3b4b64816288f887e6aa67"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "f11aefbd86bc96e3930bd2f895b9bf84"
server_key_iv: "d55836a61b75cca830176b399022c36c"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.011361 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "482124398b3b4b64816288f887e6aa67"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "f11aefbd86bc96e3930bd2f895b9bf84"
server_key_iv: "d55836a61b75cca830176b399022c36c"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.015512 5926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:10.018066 5940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.019464 5926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:10.019620 5926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "482124398b3b4b64816288f887e6aa67"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "f11aefbd86bc96e3930bd2f895b9bf84"
server_key_iv: "d55836a61b75cca830176b399022c36c"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.019747 5926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:10.024509 5926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:10.025149 5926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:10.025333 5926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:10.032236 5926 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39469
I20260619 22:21:10.032253 5992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39469 every 8 connection(s)
I20260619 22:21:10.033232 5926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:10.036564 5993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:10.039566 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 5926
I20260619 22:21:10.039695 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:10.039958 1322 external_mini_cluster.cc:1428] Setting key db30c597ac96bcc9b921f8d2bf9395ae
I20260619 22:21:10.042665 5993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67: Bootstrap starting.
I20260619 22:21:10.045069 5993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:10.045745 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.041431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.1.74.190:39469)
0619 22:21:10.042155 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.042171 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.042211 (+ 40us) server_negotiation.cc:407] Connection header received
0619 22:21:10.042943 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.042961 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.043323 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.043666 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.044446 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.044450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.044462 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.044472 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.044528 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.044543 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.044656 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.044759 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.045009 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":149,"threads_started":1}
I20260619 22:21:10.045827 5993 log.cc:824] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:10.047864 5993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67: No bootstrap required, opened a new log
I20260619 22:21:10.051448 5993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482124398b3b4b64816288f887e6aa67" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39469 } }
I20260619 22:21:10.051676 5993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:10.051708 5993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482124398b3b4b64816288f887e6aa67, State: Initialized, Role: FOLLOWER
I20260619 22:21:10.052112 5993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482124398b3b4b64816288f887e6aa67" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39469 } }
I20260619 22:21:10.052244 5993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:10.052285 5993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:10.052348 5993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:10.053289 5993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482124398b3b4b64816288f887e6aa67" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39469 } }
I20260619 22:21:10.053588 5993 leader_election.cc:302] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 482124398b3b4b64816288f887e6aa67; no voters:
I20260619 22:21:10.053853 5993 leader_election.cc:288] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:10.053994 5998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:10.054580 5998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [term 1 LEADER]: Becoming Leader. State: Replica: 482124398b3b4b64816288f887e6aa67, State: Running, Role: LEADER
I20260619 22:21:10.054993 5998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482124398b3b4b64816288f887e6aa67" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39469 } }
I20260619 22:21:10.055166 5993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:10.056687 6000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482124398b3b4b64816288f887e6aa67. Latest consensus state: current_term: 1 leader_uuid: "482124398b3b4b64816288f887e6aa67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482124398b3b4b64816288f887e6aa67" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39469 } } }
I20260619 22:21:10.056743 5999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "482124398b3b4b64816288f887e6aa67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "482124398b3b4b64816288f887e6aa67" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39469 } } }
I20260619 22:21:10.056836 6000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:10.056881 5999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:10.057266 6011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:10.060423 6011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:10.066124 6011 catalog_manager.cc:1347] Generated new cluster ID: b4f3ce0e3ca540478c702e26d6e702ec
I20260619 22:21:10.066202 6011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:10.093940 6011 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:10.094343 6011 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:10.105531 6011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67: Generated new TSK 0
I20260619 22:21:10.106247 6011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:10.116674 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39469
--builtin_ntp_servers=127.1.74.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 22:21:10.242198 6017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:10.242480 6017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:10.242528 6017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:10.246552 6017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:10.246608 6017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:10.246665 6017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:10.251412 6017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:10.252499 6017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:10.253937 6017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:10.263342 6025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.263435 6022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.263435 6023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:10.263703 6017 server_base.cc:1034] running on GCE node
I20260619 22:21:10.264091 6017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:10.264729 6017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:10.265920 6017 hybrid_clock.cc:648] HybridClock initialized: now 1781907670265904 us; error 30 us; skew 500 ppm
I20260619 22:21:10.270790 6017 webserver.cc:460] Webserver started at http://127.1.74.129:33105/ using document root <none> and password file <none>
I20260619 22:21:10.271427 6017 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:10.271521 6017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:10.271749 6017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:10.273996 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "f769fa101a7a493eaaf3a3b647877a4d"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "38de0f6c425eea3f1a98784f05c96452"
server_key_iv: "69bc9b77ee9abcf15347c8778c882614"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.274571 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "f769fa101a7a493eaaf3a3b647877a4d"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "38de0f6c425eea3f1a98784f05c96452"
server_key_iv: "69bc9b77ee9abcf15347c8778c882614"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.279426 6017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260619 22:21:10.281790 6031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.283113 6017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:10.283236 6017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "f769fa101a7a493eaaf3a3b647877a4d"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "38de0f6c425eea3f1a98784f05c96452"
server_key_iv: "69bc9b77ee9abcf15347c8778c882614"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.283347 6017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:10.287794 6017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:10.288395 6017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:10.288565 6017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:10.289101 6017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:10.289969 6017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:10.290033 6017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.290099 6017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:10.290145 6017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.298784 6017 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:42467
I20260619 22:21:10.298803 6144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:42467 every 8 connection(s)
I20260619 22:21:10.299710 6017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:10.303717 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6017
I20260619 22:21:10.303848 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:10.304138 1322 external_mini_cluster.cc:1428] Setting key 12f425466874c01530b252652fe34e78
I20260619 22:21:10.306010 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.301472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:55599 (local address 127.1.74.190:39469)
0619 22:21:10.301609 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.301613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.302688 (+ 1075us) server_negotiation.cc:407] Connection header received
0619 22:21:10.304209 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.304212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.304259 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.304343 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.305540 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.305544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.305546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.305558 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.305581 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.305594 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.305680 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.305832 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.305877 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:21:10.306340 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39469
--builtin_ntp_servers=127.1.74.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 22:21:10.307049 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.301867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39469 (local address 127.1.74.129:55599)
0619 22:21:10.302535 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.302577 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.303991 (+ 1414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.304550 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.304558 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.304649 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.305226 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.305228 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.305282 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.305373 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.305679 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.305689 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.305698 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.305881 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.306144 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
I20260619 22:21:10.308609 6145 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39469
I20260619 22:21:10.308851 6145 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:10.309252 6145 heartbeater.cc:511] Master 127.1.74.190:39469 requested a full tablet report, sending...
I20260619 22:21:10.310979 5957 ts_manager.cc:194] Registered new tserver with Master: f769fa101a7a493eaaf3a3b647877a4d (127.1.74.129:42467)
W20260619 22:21:10.416602 6148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:10.416918 6148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:10.416962 6148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:10.421155 6148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:10.421229 6148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:10.421288 6148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:10.426209 6148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:10.427381 6148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:10.428886 6148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:10.439078 6148 server_base.cc:1034] running on GCE node
W20260619 22:21:10.439096 6153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.439103 6154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.439262 6156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:10.439738 6148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:10.440414 6148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:10.441581 6148 hybrid_clock.cc:648] HybridClock initialized: now 1781907670441556 us; error 39 us; skew 500 ppm
I20260619 22:21:10.443735 6148 webserver.cc:460] Webserver started at http://127.1.74.130:36819/ using document root <none> and password file <none>
I20260619 22:21:10.444350 6148 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:10.444414 6148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:10.444662 6148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:10.446810 6148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "14eda97571e84e6a9b0823c6a8db2204"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "131cf77b0be957764e17bce54ef85c92"
server_key_iv: "25422bed004cb9b90c0cce22d83db926"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.447454 6148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "14eda97571e84e6a9b0823c6a8db2204"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "131cf77b0be957764e17bce54ef85c92"
server_key_iv: "25422bed004cb9b90c0cce22d83db926"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.451503 6148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:10.454254 6162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.455753 6148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:10.455946 6148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "14eda97571e84e6a9b0823c6a8db2204"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "131cf77b0be957764e17bce54ef85c92"
server_key_iv: "25422bed004cb9b90c0cce22d83db926"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.456122 6148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:10.461099 6148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:10.461736 6148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:10.461930 6148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:10.462548 6148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:10.463572 6148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:10.463639 6148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.463711 6148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:10.463752 6148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.472853 6148 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:37031
I20260619 22:21:10.472883 6275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:37031 every 8 connection(s)
I20260619 22:21:10.473848 6148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:10.480635 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.476008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:54779 (local address 127.1.74.190:39469)
0619 22:21:10.476175 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.476179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.477291 (+ 1112us) server_negotiation.cc:407] Connection header received
0619 22:21:10.478687 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.478690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.478731 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.478809 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.480186 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.480189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.480191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.480202 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.480221 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.480228 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.480312 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.480474 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.480513 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 22:21:10.481590 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.476417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39469 (local address 127.1.74.130:54779)
0619 22:21:10.477134 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.477180 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.478479 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.479084 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.479093 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.479193 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.479903 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.479905 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.479949 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.480025 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.480341 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.480348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.480352 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.480504 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.480720 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
I20260619 22:21:10.482996 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6148
I20260619 22:21:10.483042 6276 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39469
I20260619 22:21:10.483134 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:10.483326 6276 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:10.483418 1322 external_mini_cluster.cc:1428] Setting key 3936dd5121c37d5c643d96cf64d276b8
I20260619 22:21:10.483830 6276 heartbeater.cc:511] Master 127.1.74.190:39469 requested a full tablet report, sending...
I20260619 22:21:10.484925 5957 ts_manager.cc:194] Registered new tserver with Master: 14eda97571e84e6a9b0823c6a8db2204 (127.1.74.130:37031)
I20260619 22:21:10.485486 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39469
--builtin_ntp_servers=127.1.74.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 22:21:10.599944 6279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:10.600211 6279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:10.600251 6279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:10.604465 6279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:10.604521 6279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:10.604604 6279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:10.609516 6279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:10.610646 6279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:10.612131 6279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:10.621798 6287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.621816 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:10.621798 6285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:10.622015 6279 server_base.cc:1034] running on GCE node
I20260619 22:21:10.622644 6279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:10.623375 6279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:10.624564 6279 hybrid_clock.cc:648] HybridClock initialized: now 1781907670624546 us; error 30 us; skew 500 ppm
I20260619 22:21:10.626492 6279 webserver.cc:460] Webserver started at http://127.1.74.131:36119/ using document root <none> and password file <none>
I20260619 22:21:10.627125 6279 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:10.627214 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:10.627441 6279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:10.629648 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "249890f6d5d44cf3979dd79e3a35aca0"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "4b14a61672d3c1d47e8ecfd2922ae3f5"
server_key_iv: "a6d04f5d90fafb9c61e67f8a1e31b7bf"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.630220 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "249890f6d5d44cf3979dd79e3a35aca0"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "4b14a61672d3c1d47e8ecfd2922ae3f5"
server_key_iv: "a6d04f5d90fafb9c61e67f8a1e31b7bf"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.633870 6279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 22:21:10.636310 6293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.637836 6279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:10.638010 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "249890f6d5d44cf3979dd79e3a35aca0"
format_stamp: "Formatted at 2026-06-19 22:21:10 on dist-test-slave-k84h"
server_key: "4b14a61672d3c1d47e8ecfd2922ae3f5"
server_key_iv: "a6d04f5d90fafb9c61e67f8a1e31b7bf"
server_key_version: "encryptionkey@0"
I20260619 22:21:10.638151 6279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:10.643684 6279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:10.644421 6279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:10.644639 6279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:10.645256 6279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:10.646274 6279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:10.646358 6279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.646438 6279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:10.646497 6279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:10.656780 6279 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:44961
I20260619 22:21:10.656807 6406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:44961 every 8 connection(s)
I20260619 22:21:10.657847 6279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:10.662142 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6279
I20260619 22:21:10.662274 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:10.662575 1322 external_mini_cluster.cc:1428] Setting key 613e8c3c58f9ebfe54a4e5f8b800c9df
I20260619 22:21:10.664677 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.659833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:51385 (local address 127.1.74.190:39469)
0619 22:21:10.659996 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.660002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.661202 (+ 1200us) server_negotiation.cc:407] Connection header received
0619 22:21:10.662555 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.662559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.662608 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.662707 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.664185 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.664188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.664190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.664200 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.664218 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.664226 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.664295 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.664480 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.664528 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 22:21:10.665573 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.660290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39469 (local address 127.1.74.131:51385)
0619 22:21:10.661039 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.661099 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.662328 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.663020 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.663027 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.663121 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.663906 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.663909 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.663961 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.664053 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.664305 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.664313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.664319 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.664498 (+ 179us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.664751 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":103,"threads_started":1}
I20260619 22:21:10.666622 6407 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39469
I20260619 22:21:10.666828 6407 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:10.667279 6407 heartbeater.cc:511] Master 127.1.74.190:39469 requested a full tablet report, sending...
I20260619 22:21:10.668290 5957 ts_manager.cc:194] Registered new tserver with Master: 249890f6d5d44cf3979dd79e3a35aca0 (127.1.74.131:44961)
I20260619 22:21:10.676805 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:21:10.680465 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.678774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.1.74.190:39469)
0619 22:21:10.678937 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.678940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.679125 (+ 185us) server_negotiation.cc:407] Connection header received
0619 22:21:10.679199 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.679202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.679244 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.679330 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.680038 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.680042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.680044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.680056 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.680079 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.680090 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.680177 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.680275 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.680347 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:10.683328 5957 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:10.686420 5957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:10.698079 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.694580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:42467 (local address 127.0.0.1:46636)
0619 22:21:10.695672 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.695689 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.695774 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.696623 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.696628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.696646 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.697164 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.697168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.697195 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.697281 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.697680 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.697695 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.697706 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.697842 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.697915 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":3,"thread_start_us":128,"threads_started":1}
I20260619 22:21:10.698087 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.695317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.1.74.129:42467)
0619 22:21:10.695865 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.695876 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.695897 (+ 21us) server_negotiation.cc:407] Connection header received
0619 22:21:10.695947 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.695952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.696107 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.696620 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.697421 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.697427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.697465 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.697494 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.697543 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.697576 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.697693 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.697853 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.697924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":126,"threads_started":1}
I20260619 22:21:10.698312 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.694564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:44961 (local address 127.0.0.1:53110)
0619 22:21:10.694927 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.694979 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.695088 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.696191 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.696197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.696282 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.697415 (+ 1133us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.697416 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.697424 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.697441 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.697751 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.697757 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.697760 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.697834 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.697907 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260619 22:21:10.698665 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.695330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:37031 (local address 127.0.0.1:37148)
0619 22:21:10.696322 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.696337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.696450 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.696907 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.696913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.696929 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.697974 (+ 1045us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.697975 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.697983 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.698005 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.698409 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.698414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.698420 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.698508 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.698542 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":108,"threads_started":1}
I20260619 22:21:10.699061 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.694911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.1.74.131:44961)
0619 22:21:10.695283 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.695299 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.695322 (+ 23us) server_negotiation.cc:407] Connection header received
0619 22:21:10.695375 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.695380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.695575 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.695716 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.697549 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.697554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.697562 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.697573 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.697612 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.697648 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.697741 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.698753 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.698834 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":143,"threads_started":1}
I20260619 22:21:10.699537 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.695511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.1.74.130:37031)
0619 22:21:10.696076 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.696092 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.696434 (+ 342us) server_negotiation.cc:407] Connection header received
0619 22:21:10.696610 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.696615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.696768 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.696891 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.698153 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.698162 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.698172 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.698189 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.698245 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.698286 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.698427 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.699289 (+ 862us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.699348 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":122,"threads_started":1}
I20260619 22:21:10.701741 6341 tablet_service.cc:1467] Processing CreateTablet for tablet 24d9fef8b6204cb88a1bddf76efefde1 (DEFAULT_TABLE table=test-table [id=6d77c270828840d388caac2e87580a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:10.702112 6079 tablet_service.cc:1467] Processing CreateTablet for tablet 24d9fef8b6204cb88a1bddf76efefde1 (DEFAULT_TABLE table=test-table [id=6d77c270828840d388caac2e87580a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:10.702112 6210 tablet_service.cc:1467] Processing CreateTablet for tablet 24d9fef8b6204cb88a1bddf76efefde1 (DEFAULT_TABLE table=test-table [id=6d77c270828840d388caac2e87580a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:10.702785 6341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d9fef8b6204cb88a1bddf76efefde1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:10.703227 6210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d9fef8b6204cb88a1bddf76efefde1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:10.703228 6079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d9fef8b6204cb88a1bddf76efefde1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:10.709071 6424 tablet_bootstrap.cc:492] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Bootstrap starting.
I20260619 22:21:10.710985 6425 tablet_bootstrap.cc:492] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Bootstrap starting.
I20260619 22:21:10.711637 6426 tablet_bootstrap.cc:492] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: Bootstrap starting.
I20260619 22:21:10.712203 6424 tablet_bootstrap.cc:654] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:10.713119 6424 log.cc:824] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:10.713169 6425 tablet_bootstrap.cc:654] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:10.713773 6426 tablet_bootstrap.cc:654] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:10.713905 6425 log.cc:824] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:10.714644 6426 log.cc:824] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:10.715341 6424 tablet_bootstrap.cc:492] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: No bootstrap required, opened a new log
I20260619 22:21:10.715548 6424 ts_tablet_manager.cc:1397] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 22:21:10.715967 6425 tablet_bootstrap.cc:492] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: No bootstrap required, opened a new log
I20260619 22:21:10.716140 6425 ts_tablet_manager.cc:1397] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 22:21:10.716670 6426 tablet_bootstrap.cc:492] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: No bootstrap required, opened a new log
I20260619 22:21:10.716883 6426 ts_tablet_manager.cc:1397] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 22:21:10.720199 6424 raft_consensus.cc:348] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.720495 6424 raft_consensus.cc:374] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:10.720562 6424 raft_consensus.cc:729] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 249890f6d5d44cf3979dd79e3a35aca0, State: Initialized, Role: FOLLOWER
I20260619 22:21:10.720448 6425 raft_consensus.cc:348] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.720675 6425 raft_consensus.cc:374] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:10.720734 6425 raft_consensus.cc:729] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14eda97571e84e6a9b0823c6a8db2204, State: Initialized, Role: FOLLOWER
I20260619 22:21:10.721064 6424 consensus_queue.cc:260] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.721092 6425 consensus_queue.cc:260] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.720991 6426 raft_consensus.cc:348] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.721235 6426 raft_consensus.cc:374] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:10.721297 6426 raft_consensus.cc:729] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f769fa101a7a493eaaf3a3b647877a4d, State: Initialized, Role: FOLLOWER
I20260619 22:21:10.721719 6426 consensus_queue.cc:260] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.722112 6425 ts_tablet_manager.cc:1428] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:21:10.722112 6424 ts_tablet_manager.cc:1428] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:21:10.722465 6276 heartbeater.cc:503] Master 127.1.74.190:39469 was elected leader, sending a full tablet report...
I20260619 22:21:10.722465 6407 heartbeater.cc:503] Master 127.1.74.190:39469 was elected leader, sending a full tablet report...
I20260619 22:21:10.722477 6145 heartbeater.cc:503] Master 127.1.74.190:39469 was elected leader, sending a full tablet report...
I20260619 22:21:10.722749 6426 ts_tablet_manager.cc:1428] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260619 22:21:10.724793 6277 tablet.cc:2369] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:21:10.800869 6146 tablet.cc:2369] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:21:10.905165 6432 raft_consensus.cc:482] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:10.905429 6432 raft_consensus.cc:504] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.906695 6432 leader_election.cc:288] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 249890f6d5d44cf3979dd79e3a35aca0 (127.1.74.131:44961), 14eda97571e84e6a9b0823c6a8db2204 (127.1.74.130:37031)
I20260619 22:21:10.908934 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.906962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:44961 (local address 127.1.74.129:60961)
0619 22:21:10.907222 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.907238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.907355 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.907744 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.907747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.907762 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.908307 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.908309 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.908319 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.908343 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.908617 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.908623 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.908630 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.908705 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.908760 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":90,"threads_started":1}
W20260619 22:21:10.908901 6408 tablet.cc:2369] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:21:10.908939 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.907058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:60961 (local address 127.1.74.131:44961)
0619 22:21:10.907200 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.907205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.907328 (+ 123us) server_negotiation.cc:407] Connection header received
0619 22:21:10.907489 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.907494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.907532 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.907625 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.908471 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.908478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.908483 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.908497 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.908524 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.908533 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.908633 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.908727 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.908771 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 22:21:10.909180 6423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.907279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:53925 (local address 127.1.74.130:37031)
0619 22:21:10.907415 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.907422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.907526 (+ 104us) server_negotiation.cc:407] Connection header received
0619 22:21:10.907731 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.907734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.907772 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.907861 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.908693 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.908697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.908699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.908712 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.908734 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.908745 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.908828 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.908983 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.909034 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:21:10.909508 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.907054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:37031 (local address 127.1.74.129:53925)
0619 22:21:10.907415 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.907430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:10.907523 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:10.907901 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:10.907904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:10.907927 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:10.908524 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:10.908525 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:10.908533 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:10.908552 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:10.908871 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:10.908874 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:10.908877 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 22:21:10.909326 (+ 449us) client_negotiation.cc:240] Negotiation successful
0619 22:21:10.909365 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":65,"threads_started":1}
I20260619 22:21:10.909689 6361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d9fef8b6204cb88a1bddf76efefde1" candidate_uuid: "f769fa101a7a493eaaf3a3b647877a4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "249890f6d5d44cf3979dd79e3a35aca0" is_pre_election: true
I20260619 22:21:10.910008 6361 raft_consensus.cc:2454] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f769fa101a7a493eaaf3a3b647877a4d in term 0.
I20260619 22:21:10.910096 6230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d9fef8b6204cb88a1bddf76efefde1" candidate_uuid: "f769fa101a7a493eaaf3a3b647877a4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14eda97571e84e6a9b0823c6a8db2204" is_pre_election: true
I20260619 22:21:10.910387 6230 raft_consensus.cc:2454] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f769fa101a7a493eaaf3a3b647877a4d in term 0.
I20260619 22:21:10.910468 6032 leader_election.cc:302] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 249890f6d5d44cf3979dd79e3a35aca0, f769fa101a7a493eaaf3a3b647877a4d; no voters:
I20260619 22:21:10.910763 6432 raft_consensus.cc:2781] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:10.910885 6432 raft_consensus.cc:482] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:10.910944 6432 raft_consensus.cc:3037] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:10.911994 6432 raft_consensus.cc:504] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.912381 6432 leader_election.cc:288] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [CANDIDATE]: Term 1 election: Requested vote from peers 249890f6d5d44cf3979dd79e3a35aca0 (127.1.74.131:44961), 14eda97571e84e6a9b0823c6a8db2204 (127.1.74.130:37031)
I20260619 22:21:10.912717 6361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d9fef8b6204cb88a1bddf76efefde1" candidate_uuid: "f769fa101a7a493eaaf3a3b647877a4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "249890f6d5d44cf3979dd79e3a35aca0"
I20260619 22:21:10.912787 6230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d9fef8b6204cb88a1bddf76efefde1" candidate_uuid: "f769fa101a7a493eaaf3a3b647877a4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14eda97571e84e6a9b0823c6a8db2204"
I20260619 22:21:10.912837 6361 raft_consensus.cc:3037] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:10.912947 6230 raft_consensus.cc:3037] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:10.913932 6361 raft_consensus.cc:2454] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f769fa101a7a493eaaf3a3b647877a4d in term 1.
I20260619 22:21:10.914027 6230 raft_consensus.cc:2454] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f769fa101a7a493eaaf3a3b647877a4d in term 1.
I20260619 22:21:10.914263 6032 leader_election.cc:302] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 249890f6d5d44cf3979dd79e3a35aca0, f769fa101a7a493eaaf3a3b647877a4d; no voters:
I20260619 22:21:10.914448 6432 raft_consensus.cc:2781] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:10.914676 6432 raft_consensus.cc:686] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [term 1 LEADER]: Becoming Leader. State: Replica: f769fa101a7a493eaaf3a3b647877a4d, State: Running, Role: LEADER
I20260619 22:21:10.915045 6432 consensus_queue.cc:237] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } }
I20260619 22:21:10.918362 5955 catalog_manager.cc:5495] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d reported cstate change: term changed from 0 to 1, leader changed from <none> to f769fa101a7a493eaaf3a3b647877a4d (127.1.74.129). New cstate: current_term: 1 leader_uuid: "f769fa101a7a493eaaf3a3b647877a4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f769fa101a7a493eaaf3a3b647877a4d" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 42467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 } health_report { overall_health: UNKNOWN } } }
I20260619 22:21:10.935423 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.934065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.1.74.129:42467)
0619 22:21:10.934187 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:10.934190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:10.934202 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:10.934282 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:10.934284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:10.934343 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:10.934414 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:10.935100 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:10.935103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:10.935105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:10.935116 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:10.935135 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:10.935144 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:10.935218 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:10.935271 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 22:21:10.935305 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 22:21:10.941519 6230 raft_consensus.cc:1261] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 1 FOLLOWER]: Refusing update from remote peer f769fa101a7a493eaaf3a3b647877a4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:10.941519 6361 raft_consensus.cc:1261] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 1 FOLLOWER]: Refusing update from remote peer f769fa101a7a493eaaf3a3b647877a4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:10.942276 6435 consensus_queue.cc:1035] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "249890f6d5d44cf3979dd79e3a35aca0" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:10.942481 6432 consensus_queue.cc:1035] T 24d9fef8b6204cb88a1bddf76efefde1 P f769fa101a7a493eaaf3a3b647877a4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "14eda97571e84e6a9b0823c6a8db2204" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:10.951207 6440 mvcc.cc:205] Tried to move back new op lower bound from 7298693820168593408 to 7298693820070805504. Current Snapshot: MvccSnapshot[applied={T|T < 7298693820168593408}]
I20260619 22:21:10.952746 6439 mvcc.cc:205] Tried to move back new op lower bound from 7298693820168593408 to 7298693820070805504. Current Snapshot: MvccSnapshot[applied={T|T < 7298693820168593408}]
I20260619 22:21:10.959358 5955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:10.959585 5955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:10.962203 5955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 482124398b3b4b64816288f887e6aa67: Sending DeleteTablet for 3 replicas of tablet 24d9fef8b6204cb88a1bddf76efefde1
I20260619 22:21:10.963047 6079 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d9fef8b6204cb88a1bddf76efefde1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:10 UTC) from {username='slave'} at 127.0.0.1:46636
I20260619 22:21:10.963093 6341 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d9fef8b6204cb88a1bddf76efefde1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:10 UTC) from {username='slave'} at 127.0.0.1:53110
I20260619 22:21:10.963244 6210 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d9fef8b6204cb88a1bddf76efefde1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:10 UTC) from {username='slave'} at 127.0.0.1:37148
I20260619 22:21:10.963666 6450 tablet_replica.cc:330] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: stopping tablet replica
I20260619 22:21:10.963707 6449 tablet_replica.cc:330] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: stopping tablet replica
I20260619 22:21:10.964013 6450 raft_consensus.cc:2229] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:10.964015 6449 raft_consensus.cc:2229] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:10.964344 6450 raft_consensus.cc:2258] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:10.964347 6449 raft_consensus.cc:2258] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:10.964502 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6017
I20260619 22:21:10.966152 6450 ts_tablet_manager.cc:1905] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:10.966182 6449 ts_tablet_manager.cc:1905] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:10.969298 6449 ts_tablet_manager.cc:1918] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:10.969318 6450 ts_tablet_manager.cc:1918] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:10.969415 6449 log.cc:1196] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/24d9fef8b6204cb88a1bddf76efefde1
I20260619 22:21:10.969414 6450 log.cc:1196] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/24d9fef8b6204cb88a1bddf76efefde1
I20260619 22:21:10.969725 6450 ts_tablet_manager.cc:1939] T 24d9fef8b6204cb88a1bddf76efefde1 P 14eda97571e84e6a9b0823c6a8db2204: Deleting consensus metadata
I20260619 22:21:10.969709 6449 ts_tablet_manager.cc:1939] T 24d9fef8b6204cb88a1bddf76efefde1 P 249890f6d5d44cf3979dd79e3a35aca0: Deleting consensus metadata
I20260619 22:21:10.970947 5941 catalog_manager.cc:4841] TS 249890f6d5d44cf3979dd79e3a35aca0 (127.1.74.131:44961): tablet 24d9fef8b6204cb88a1bddf76efefde1 (table test-table [id=6d77c270828840d388caac2e87580a9e]) successfully deleted
I20260619 22:21:10.970947 5942 catalog_manager.cc:4841] TS 14eda97571e84e6a9b0823c6a8db2204 (127.1.74.130:37031): tablet 24d9fef8b6204cb88a1bddf76efefde1 (table test-table [id=6d77c270828840d388caac2e87580a9e]) successfully deleted
I20260619 22:21:10.973042 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6148
W20260619 22:21:10.973034 5943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:42467 (error 108)
I20260619 22:21:10.975198 6419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:10.974819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:42467 (local address 127.0.0.1:46658)
0619 22:21:10.975001 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:10.975087 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:42467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260619 22:21:10.975376 5943 catalog_manager.cc:4584] TS f769fa101a7a493eaaf3a3b647877a4d (127.1.74.129:42467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24d9fef8b6204cb88a1bddf76efefde1: Network error: Client connection negotiation failed: client connection to 127.1.74.129:42467: connect: Connection refused (error 111)
I20260619 22:21:10.978907 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6279
I20260619 22:21:10.984159 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 5926
2026-06-19T22:21:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1150 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 22:21:11.005959 1322 mini_oidc.cc:152] Starting JWKS server
I20260619 22:21:11.007640 1322 webserver.cc:460] Webserver started at https://127.0.0.1:43105/ using document root <none> and password file <none>
I20260619 22:21:11.007786 1322 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 22:21:11.008023 1322 webserver.cc:460] Webserver started at http://127.0.0.1:45275/ using document root <none> and password file <none>
2026-06-19T22:21:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:11Z Disabled control of system clock
I20260619 22:21:11.017617 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39111
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:34055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39111
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43105/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/testchainca.cert with env {}
W20260619 22:21:11.128728 6457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:11.129071 6457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:11.129168 6457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:11.133277 6457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:11.133379 6457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:11.133409 6457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:11.133446 6457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:11.133474 6457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 22:21:11.133512 6457 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 22:21:11.133544 6457 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43105/jwks.json
I20260619 22:21:11.138985 6457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39111
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43105/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:11.140297 6457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:11.141361 6457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:11.150394 6462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:11.150498 6463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:11.150614 6465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:11.150717 6457 server_base.cc:1034] running on GCE node
I20260619 22:21:11.151319 6457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:11.152477 6457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:11.153695 6457 hybrid_clock.cc:648] HybridClock initialized: now 1781907671153679 us; error 44 us; skew 500 ppm
I20260619 22:21:11.155710 6457 webserver.cc:460] Webserver started at http://127.1.74.190:46493/ using document root <none> and password file <none>
I20260619 22:21:11.156297 6457 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:11.156384 6457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:11.156605 6457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:11.158699 6457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "cf442283cc03483e8558de93a22d2861"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "085fb6d3d6a403549befaca844cce008"
server_key_iv: "76d37e0d067a44ae3447beb869ff9298"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.159338 6457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "cf442283cc03483e8558de93a22d2861"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "085fb6d3d6a403549befaca844cce008"
server_key_iv: "76d37e0d067a44ae3447beb869ff9298"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.163090 6457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:11.165534 6471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:11.166774 6457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:11.166947 6457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "cf442283cc03483e8558de93a22d2861"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "085fb6d3d6a403549befaca844cce008"
server_key_iv: "76d37e0d067a44ae3447beb869ff9298"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.167078 6457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:11.185336 6457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:11.186087 6457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:11.186290 6457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:11.193190 6457 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39111
I20260619 22:21:11.193200 6523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39111 every 8 connection(s)
I20260619 22:21:11.194211 6457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:11.197476 6524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:11.203644 6524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861: Bootstrap starting.
I20260619 22:21:11.203804 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6457
I20260619 22:21:11.203938 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:11.204192 1322 external_mini_cluster.cc:1428] Setting key 22759cf9fc8e297eb1c586826ee6ca22
I20260619 22:21:11.206308 6524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:11.207123 6524 log.cc:824] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:11.209179 6524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861: No bootstrap required, opened a new log
I20260619 22:21:11.212853 6524 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf442283cc03483e8558de93a22d2861" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39111 } }
I20260619 22:21:11.213222 6524 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:11.213306 6524 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf442283cc03483e8558de93a22d2861, State: Initialized, Role: FOLLOWER
I20260619 22:21:11.213773 6524 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf442283cc03483e8558de93a22d2861" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39111 } }
I20260619 22:21:11.213924 6524 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:11.214007 6524 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:11.214073 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:11.205590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47660 (local address 127.1.74.190:39111)
0619 22:21:11.206300 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:11.206322 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:11.206372 (+ 50us) server_negotiation.cc:407] Connection header received
0619 22:21:11.207150 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:11.207170 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:11.207651 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:11.208030 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:11.208476 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.209506 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.210411 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.210985 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.211157 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.212013 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:11.212024 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:11.212037 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:11.212456 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:11.212513 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:11.212534 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:11.212634 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:11.212795 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 22:21:11.213079 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":137,"threads_started":1}
I20260619 22:21:11.214124 6524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:11.215425 6524 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf442283cc03483e8558de93a22d2861" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39111 } }
I20260619 22:21:11.215780 6524 leader_election.cc:302] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf442283cc03483e8558de93a22d2861; no voters:
I20260619 22:21:11.216153 6524 leader_election.cc:288] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:11.216512 6529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:11.216711 6529 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [term 1 LEADER]: Becoming Leader. State: Replica: cf442283cc03483e8558de93a22d2861, State: Running, Role: LEADER
I20260619 22:21:11.217023 6529 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf442283cc03483e8558de93a22d2861" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39111 } }
I20260619 22:21:11.217231 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:21:11.217514 6524 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:11.218169 6529 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf442283cc03483e8558de93a22d2861. Latest consensus state: current_term: 1 leader_uuid: "cf442283cc03483e8558de93a22d2861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf442283cc03483e8558de93a22d2861" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39111 } } }
I20260619 22:21:11.218282 6529 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:11.219007 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf442283cc03483e8558de93a22d2861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf442283cc03483e8558de93a22d2861" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39111 } } }
I20260619 22:21:11.219103 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:11.219105 6535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:11.222611 6535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:11.228793 6535 catalog_manager.cc:1347] Generated new cluster ID: 514af7e25e494150a58bfeb360e0f4db
I20260619 22:21:11.228899 6535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:11.254768 6535 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:11.255791 6535 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:11.261845 6535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf442283cc03483e8558de93a22d2861: Generated new TSK 0
I20260619 22:21:11.262815 6535 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 22:21:11.296514 6554 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 22:21:11.296655 6554 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 22:21:11.297341 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:11.278617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47672 (local address 127.1.74.190:39111)
0619 22:21:11.278756 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:11.278760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:11.278968 (+ 208us) server_negotiation.cc:407] Connection header received
0619 22:21:11.279258 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:11.279261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:11.279369 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:11.279483 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0619 22:21:11.279840 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.280657 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.282200 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.282573 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.282681 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.283107 (+ 426us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 22:21:11.296855 (+ 13748us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43105/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 22:21:11.297176 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47672: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43105/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 22:21:11.297446 6527 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47672: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43105/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 22:21:11.298552 6553 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:11.278492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39111 (local address 127.0.0.1:47672)
0619 22:21:11.278878 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:11.278895 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:11.279056 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:11.279528 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:11.279532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:11.279541 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 22:21:11.279687 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:11.279694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.280831 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.280835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:11.282025 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:11.282035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.282702 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.282706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:11.282804 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.282896 (+ 92us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 22:21:11.297106 (+ 14210us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43105/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 22:21:11.297357 (+ 251us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.74.190:39111: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43105/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.74.190:39111: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43105/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 22:21:11.299283 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6457
2026-06-19T22:21:11Z chronyd exiting
I20260619 22:21:11.336968 1322 test_util.cc:175] -----------------------------------------------
I20260619 22:21:11.337064 1322 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781907643212257-1322-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (332 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 22:21:11.338075 1322 mini_oidc.cc:152] Starting JWKS server
I20260619 22:21:11.338991 1322 webserver.cc:460] Webserver started at https://127.0.0.1:45417/ using document root <none> and password file <none>
I20260619 22:21:11.339092 1322 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 22:21:11.339310 1322 webserver.cc:460] Webserver started at http://127.0.0.1:45077/ using document root <none> and password file <none>
2026-06-19T22:21:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:11Z Disabled control of system clock
I20260619 22:21:11.348776 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39325
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:36471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39325
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45417/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/testchainca.cert with env {}
W20260619 22:21:11.460623 6560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:11.460912 6560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:11.460961 6560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:11.465065 6560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:11.465126 6560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:11.465142 6560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:11.465160 6560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:11.465175 6560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 22:21:11.465191 6560 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 22:21:11.465205 6560 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45417/jwks.json
I20260619 22:21:11.470537 6560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:36471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45417/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:11.471645 6560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:11.472637 6560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:11.481353 6560 server_base.cc:1034] running on GCE node
W20260619 22:21:11.481304 6566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:11.481333 6565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:11.481304 6568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:11.482113 6560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:11.483258 6560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:11.484408 6560 hybrid_clock.cc:648] HybridClock initialized: now 1781907671484394 us; error 26 us; skew 500 ppm
I20260619 22:21:11.486261 6560 webserver.cc:460] Webserver started at http://127.1.74.190:41309/ using document root <none> and password file <none>
I20260619 22:21:11.486901 6560 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:11.486968 6560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:11.487185 6560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:11.489197 6560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "8289754508224449a1c0d68a63c3c935"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "fa2eed1468fa85aef225b49cd3e9ebd8"
server_key_iv: "4e527ede5c6dee3dffbf64dad724aca8"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.489749 6560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "8289754508224449a1c0d68a63c3c935"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "fa2eed1468fa85aef225b49cd3e9ebd8"
server_key_iv: "4e527ede5c6dee3dffbf64dad724aca8"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.493306 6560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260619 22:21:11.495721 6574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:11.496901 6560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:11.497030 6560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "8289754508224449a1c0d68a63c3c935"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "fa2eed1468fa85aef225b49cd3e9ebd8"
server_key_iv: "4e527ede5c6dee3dffbf64dad724aca8"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.497136 6560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:11.534221 6560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:11.535038 6560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:11.535244 6560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:11.542287 6560 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39325
I20260619 22:21:11.542346 6626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39325 every 8 connection(s)
I20260619 22:21:11.543344 6560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:11.545254 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6560
I20260619 22:21:11.545392 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:11.545678 1322 external_mini_cluster.cc:1428] Setting key d004c73e42d0af84d80f9eb6f9c3c1f2
I20260619 22:21:11.546651 6627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:11.553258 6627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935: Bootstrap starting.
I20260619 22:21:11.555076 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:11.547076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.1.74.190:39325)
0619 22:21:11.547686 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:11.547705 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:11.547741 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:21:11.548397 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:11.548413 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:11.548801 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:11.549090 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:11.549420 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.550383 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.551613 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.552095 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.552215 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.553174 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:11.553179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:11.553191 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:11.553671 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:11.553727 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:11.553748 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:11.553827 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:11.553991 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 22:21:11.554281 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":137,"threads_started":1}
I20260619 22:21:11.555783 6627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:11.556571 6627 log.cc:824] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:11.557919 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:21:11.559013 6627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935: No bootstrap required, opened a new log
I20260619 22:21:11.562623 6627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8289754508224449a1c0d68a63c3c935" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39325 } }
I20260619 22:21:11.562906 6627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:11.562974 6627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8289754508224449a1c0d68a63c3c935, State: Initialized, Role: FOLLOWER
I20260619 22:21:11.563391 6627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8289754508224449a1c0d68a63c3c935" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39325 } }
I20260619 22:21:11.563549 6627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:11.563601 6627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:11.563673 6627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:11.564612 6627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8289754508224449a1c0d68a63c3c935" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39325 } }
I20260619 22:21:11.564949 6627 leader_election.cc:302] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8289754508224449a1c0d68a63c3c935; no voters:
I20260619 22:21:11.565325 6627 leader_election.cc:288] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:11.565452 6632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:11.565668 6632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [term 1 LEADER]: Becoming Leader. State: Replica: 8289754508224449a1c0d68a63c3c935, State: Running, Role: LEADER
I20260619 22:21:11.565994 6632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8289754508224449a1c0d68a63c3c935" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39325 } }
I20260619 22:21:11.566570 6627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:11.567646 6633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8289754508224449a1c0d68a63c3c935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8289754508224449a1c0d68a63c3c935" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39325 } } }
I20260619 22:21:11.567773 6633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:11.568075 6634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8289754508224449a1c0d68a63c3c935. Latest consensus state: current_term: 1 leader_uuid: "8289754508224449a1c0d68a63c3c935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8289754508224449a1c0d68a63c3c935" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39325 } } }
I20260619 22:21:11.568174 6634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:11.568251 6641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:11.571568 6641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:11.577934 6641 catalog_manager.cc:1347] Generated new cluster ID: b33bb62b7e624e859a7932d10e419d51
I20260619 22:21:11.578032 6641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:11.601261 6641 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:11.602720 6641 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:11.618927 6641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8289754508224449a1c0d68a63c3c935: Generated new TSK 0
I20260619 22:21:11.619824 6641 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 22:21:11.650309 6657 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 22:21:11.650413 6657 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 22:21:11.651121 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:11.634402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.1.74.190:39325)
0619 22:21:11.634536 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:11.634541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:11.634712 (+ 171us) server_negotiation.cc:407] Connection header received
0619 22:21:11.634875 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:11.634878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:11.634935 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:11.635047 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0619 22:21:11.635252 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.636085 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.636945 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.637288 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.637398 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.637659 (+ 261us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 22:21:11.650677 (+ 13018us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45417/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 22:21:11.650972 (+ 295us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44084: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45417/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 22:21:11.651165 6656 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:11.634291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39325 (local address 127.0.0.1:44084)
0619 22:21:11.634633 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:11.634645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:11.634737 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:11.635058 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:11.635060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:11.635066 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0619 22:21:11.635156 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:11.635160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.636205 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.636207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:11.636836 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:11.636844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.637397 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.637400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:11.637473 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.637494 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 22:21:11.650892 (+ 13398us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45417/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 22:21:11.651027 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.74.190:39325: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45417/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":100,"threads_started":1}
W20260619 22:21:11.651189 6630 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44084: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45417/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260619 22:21:11.651906 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6560
2026-06-19T22:21:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 22:21:11.691052 1322 mini_oidc.cc:152] Starting JWKS server
I20260619 22:21:11.691951 1322 webserver.cc:460] Webserver started at https://127.0.0.1:37837/ using document root <none> and password file <none>
I20260619 22:21:11.692034 1322 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 22:21:11.692309 1322 webserver.cc:460] Webserver started at http://127.0.0.1:33607/ using document root <none> and password file <none>
2026-06-19T22:21:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:11Z Disabled control of system clock
I20260619 22:21:11.701388 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37475
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37475
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37837/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 22:21:11.812824 6663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:11.813104 6663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:11.813144 6663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:11.817070 6663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:11.817124 6663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:11.817139 6663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:11.817157 6663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:11.817173 6663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 22:21:11.817188 6663 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 22:21:11.817201 6663 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37837/jwks.json
I20260619 22:21:11.821868 6663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37837/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:11.822963 6663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:11.824297 6663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:11.832752 6671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:11.832791 6668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:11.832752 6669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:11.833062 6663 server_base.cc:1034] running on GCE node
I20260619 22:21:11.833802 6663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:11.835018 6663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:11.836192 6663 hybrid_clock.cc:648] HybridClock initialized: now 1781907671836179 us; error 29 us; skew 500 ppm
I20260619 22:21:11.838119 6663 webserver.cc:460] Webserver started at http://127.1.74.190:42789/ using document root <none> and password file <none>
I20260619 22:21:11.838707 6663 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:11.838795 6663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:11.839056 6663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:11.841187 6663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "d3528863b039474cb9090096d82eeeb5"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "ad308a7007bfcd227696359cc324f3ba"
server_key_iv: "dc8da026664b11aaeac8a5cf8f6d104f"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.841780 6663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "d3528863b039474cb9090096d82eeeb5"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "ad308a7007bfcd227696359cc324f3ba"
server_key_iv: "dc8da026664b11aaeac8a5cf8f6d104f"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.845757 6663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 22:21:11.848501 6677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:11.849779 6663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:11.849936 6663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "d3528863b039474cb9090096d82eeeb5"
format_stamp: "Formatted at 2026-06-19 22:21:11 on dist-test-slave-k84h"
server_key: "ad308a7007bfcd227696359cc324f3ba"
server_key_iv: "dc8da026664b11aaeac8a5cf8f6d104f"
server_key_version: "encryptionkey@0"
I20260619 22:21:11.850060 6663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:11.860639 6663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:11.861379 6663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:11.861582 6663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:11.869076 6663 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:37475
I20260619 22:21:11.869103 6729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:37475 every 8 connection(s)
I20260619 22:21:11.870190 6663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:11.873805 6730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:11.877390 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6663
I20260619 22:21:11.877543 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:11.877805 1322 external_mini_cluster.cc:1428] Setting key 871aa05a2d95e7085cbc1fb6e90ed990
I20260619 22:21:11.880530 6730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5: Bootstrap starting.
I20260619 22:21:11.883649 6730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:11.884598 6730 log.cc:824] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:11.887485 6730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5: No bootstrap required, opened a new log
I20260619 22:21:11.889649 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:11.879405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.1.74.190:37475)
0619 22:21:11.880174 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:11.880195 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:11.880241 (+ 46us) server_negotiation.cc:407] Connection header received
0619 22:21:11.881151 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:11.881177 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:11.881709 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:11.882162 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:11.882643 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.884175 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.885327 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.885998 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.886186 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.887290 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:11.887297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:11.887316 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:11.887899 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:11.887998 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:11.888027 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:11.888132 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:11.888304 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 22:21:11.888606 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":141,"threads_started":1}
I20260619 22:21:11.892428 6730 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3528863b039474cb9090096d82eeeb5" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37475 } }
I20260619 22:21:11.892751 6730 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:11.892839 6730 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3528863b039474cb9090096d82eeeb5, State: Initialized, Role: FOLLOWER
I20260619 22:21:11.893110 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:21:11.893371 6730 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3528863b039474cb9090096d82eeeb5" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37475 } }
I20260619 22:21:11.893537 6730 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:11.893627 6730 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:11.893733 6730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:11.894980 6730 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3528863b039474cb9090096d82eeeb5" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37475 } }
I20260619 22:21:11.895372 6730 leader_election.cc:302] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3528863b039474cb9090096d82eeeb5; no voters:
I20260619 22:21:11.895714 6730 leader_election.cc:288] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:11.895987 6735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:11.896387 6735 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [term 1 LEADER]: Becoming Leader. State: Replica: d3528863b039474cb9090096d82eeeb5, State: Running, Role: LEADER
I20260619 22:21:11.896834 6735 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3528863b039474cb9090096d82eeeb5" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37475 } }
I20260619 22:21:11.897264 6730 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:11.899073 6737 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3528863b039474cb9090096d82eeeb5. Latest consensus state: current_term: 1 leader_uuid: "d3528863b039474cb9090096d82eeeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3528863b039474cb9090096d82eeeb5" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37475 } } }
I20260619 22:21:11.899108 6736 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3528863b039474cb9090096d82eeeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3528863b039474cb9090096d82eeeb5" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37475 } } }
I20260619 22:21:11.899256 6737 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:11.899259 6736 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:11.899781 6747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:11.903702 6747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:11.910351 6747 catalog_manager.cc:1347] Generated new cluster ID: 4ad874deb406465f85ea2de17cf64398
I20260619 22:21:11.910463 6747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:11.932503 6747 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:11.933554 6747 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:11.941167 6747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3528863b039474cb9090096d82eeeb5: Generated new TSK 0
I20260619 22:21:11.941963 6747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:11.963579 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:11.958789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43454 (local address 127.1.74.190:37475)
0619 22:21:11.958943 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:11.958948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:11.959118 (+ 170us) server_negotiation.cc:407] Connection header received
0619 22:21:11.959324 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:11.959327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:11.959376 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:11.959497 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0619 22:21:11.959734 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.960291 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.961193 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.961419 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.961519 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.961864 (+ 345us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 22:21:11.963200 (+ 1336us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 22:21:11.963456 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43454: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260619 22:21:11.963665 6733 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43454: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 22:21:11.966536 6759 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:11.958703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37475 (local address 127.0.0.1:43454)
0619 22:21:11.959031 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:11.959044 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:11.959174 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:11.959492 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:11.959495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:11.959503 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0619 22:21:11.959623 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:11.959629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.960411 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.960414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:11.961062 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:11.961071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:11.961557 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:11.961562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:11.961652 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:11.961676 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 22:21:11.963366 (+ 1690us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 22:21:11.966366 (+ 3000us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.74.190:37475: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":2814,"thread_start_us":124,"threads_started":1}
I20260619 22:21:11.967142 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6663
E20260619 22:21:11.975493 6760 webserver.cc:533] Webserver: sslize failed:
E20260619 22:21:11.975569 6760 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T22:21:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (315 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T22:21:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:12Z Disabled control of system clock
I20260619 22:21:12.015354 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40713
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:35657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40713
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/testchainca.cert with env {}
W20260619 22:21:12.125005 6764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:12.125265 6764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:12.125331 6764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:12.129324 6764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:12.129379 6764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:12.129395 6764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:12.129457 6764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:12.129510 6764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 22:21:12.129544 6764 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 22:21:12.129573 6764 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 22:21:12.134966 6764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:12.135965 6764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:12.136937 6764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:12.145270 6772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:12.145314 6769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:12.145272 6770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:12.145432 6764 server_base.cc:1034] running on GCE node
I20260619 22:21:12.146137 6764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:12.147257 6764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:12.148458 6764 hybrid_clock.cc:648] HybridClock initialized: now 1781907672148433 us; error 35 us; skew 500 ppm
I20260619 22:21:12.150640 6764 webserver.cc:460] Webserver started at http://127.1.74.190:38627/ using document root <none> and password file <none>
I20260619 22:21:12.151320 6764 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:12.151417 6764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:12.151623 6764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:12.153726 6764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "b09e902f3b9c43ef8bca1822d5933ddf"
format_stamp: "Formatted at 2026-06-19 22:21:12 on dist-test-slave-k84h"
server_key: "c175947ad06fe8252221c55b3067a1bd"
server_key_iv: "ed1dfa22b19baf559296e39042b93f53"
server_key_version: "encryptionkey@0"
I20260619 22:21:12.154327 6764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "b09e902f3b9c43ef8bca1822d5933ddf"
format_stamp: "Formatted at 2026-06-19 22:21:12 on dist-test-slave-k84h"
server_key: "c175947ad06fe8252221c55b3067a1bd"
server_key_iv: "ed1dfa22b19baf559296e39042b93f53"
server_key_version: "encryptionkey@0"
I20260619 22:21:12.158293 6764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:21:12.160902 6778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:12.162142 6764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:12.162289 6764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "b09e902f3b9c43ef8bca1822d5933ddf"
format_stamp: "Formatted at 2026-06-19 22:21:12 on dist-test-slave-k84h"
server_key: "c175947ad06fe8252221c55b3067a1bd"
server_key_iv: "ed1dfa22b19baf559296e39042b93f53"
server_key_version: "encryptionkey@0"
I20260619 22:21:12.162413 6764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:12.189471 6764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:12.190235 6764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:12.190431 6764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:12.197297 6764 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40713
I20260619 22:21:12.197315 6830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40713 every 8 connection(s)
I20260619 22:21:12.198269 6764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:12.201217 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6764
I20260619 22:21:12.201349 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:12.201444 6831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:12.201678 1322 external_mini_cluster.cc:1428] Setting key eb5fbe50fa45c20f080bef711a4d8b97
I20260619 22:21:12.208813 6831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf: Bootstrap starting.
I20260619 22:21:12.211695 6831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:12.212548 6831 log.cc:824] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:12.212613 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:12.203062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.1.74.190:40713)
0619 22:21:12.203725 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:12.203744 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:12.203787 (+ 43us) server_negotiation.cc:407] Connection header received
0619 22:21:12.204958 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:12.204985 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:12.205503 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:12.205875 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:12.206318 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:12.207862 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:12.208804 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:12.209483 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:12.209636 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:12.210541 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:12.210547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:12.210562 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:12.211095 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:12.211162 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:12.211188 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:12.211283 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:12.211394 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 22:21:12.211673 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
I20260619 22:21:12.214623 6831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf: No bootstrap required, opened a new log
I20260619 22:21:12.215302 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:21:12.218111 6831 raft_consensus.cc:348] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40713 } }
I20260619 22:21:12.218357 6831 raft_consensus.cc:374] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:12.218425 6831 raft_consensus.cc:729] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09e902f3b9c43ef8bca1822d5933ddf, State: Initialized, Role: FOLLOWER
I20260619 22:21:12.218897 6831 consensus_queue.cc:260] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40713 } }
I20260619 22:21:12.219054 6831 raft_consensus.cc:388] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:12.219118 6831 raft_consensus.cc:482] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:12.219211 6831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:12.220125 6831 raft_consensus.cc:504] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40713 } }
I20260619 22:21:12.220463 6831 leader_election.cc:302] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b09e902f3b9c43ef8bca1822d5933ddf; no voters:
I20260619 22:21:12.220798 6831 leader_election.cc:288] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:12.220927 6836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:12.221208 6836 raft_consensus.cc:686] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [term 1 LEADER]: Becoming Leader. State: Replica: b09e902f3b9c43ef8bca1822d5933ddf, State: Running, Role: LEADER
I20260619 22:21:12.221501 6836 consensus_queue.cc:237] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40713 } }
I20260619 22:21:12.221975 6831 sys_catalog.cc:564] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:12.223296 6838 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader b09e902f3b9c43ef8bca1822d5933ddf. Latest consensus state: current_term: 1 leader_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40713 } } }
I20260619 22:21:12.223344 6837 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09e902f3b9c43ef8bca1822d5933ddf" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40713 } } }
I20260619 22:21:12.223426 6838 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:12.223448 6837 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:12.223780 6845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:12.226531 6845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:12.232298 6845 catalog_manager.cc:1347] Generated new cluster ID: 920d5484c32b451b8ed3debc3eb8f657
I20260619 22:21:12.232370 6845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:12.257262 6845 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:12.258567 6845 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:12.266795 6845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b09e902f3b9c43ef8bca1822d5933ddf: Generated new TSK 0
I20260619 22:21:12.267580 6845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:12.811070 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:12.274327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.1.74.190:40713)
0619 22:21:12.274466 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:12.274470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:12.274609 (+ 139us) server_negotiation.cc:407] Connection header received
0619 22:21:12.274752 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:12.274756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:12.274811 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:12.274941 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0619 22:21:12.275188 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:12.276003 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:12.276879 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:12.277228 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:12.277331 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:12.277674 (+ 343us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 22:21:12.810563 (+532889us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 22:21:12.810935 (+ 372us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50044: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 22:21:12.811173 6834 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50044: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 22:21:12.811232 6860 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:12.274236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40713 (local address 127.0.0.1:50044)
0619 22:21:12.274530 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:12.274543 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:12.274636 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:12.274976 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:12.274979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:12.274986 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0619 22:21:12.275079 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:12.275084 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:12.276126 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:12.276129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:12.276762 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:12.276770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:12.277377 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:12.277380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:12.277456 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:12.277481 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 22:21:12.810815 (+533334us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 22:21:12.811056 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.74.190:40713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":117,"threads_started":1}
I20260619 22:21:12.811802 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6764
2026-06-19T22:21:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (824 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (122 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:13 dist-test-slave-k84h krb5kdc[6872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:13 dist-test-slave-k84h krb5kdc[6872](info): set up 2 sockets
Jun 19 22:21:13 dist-test-slave-k84h krb5kdc[6872](info): commencing operation
krb5kdc: starting...
W20260619 22:21:14.107282 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:14.231959 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40041
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:14.343526 6885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:14.343844 6885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:14.343935 6885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:14.347966 6885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:14.348054 6885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:14.348095 6885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:14.348145 6885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:14.348184 6885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:14.353521 6885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:14.354718 6885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:14.355801 6885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:14.364439 6885 server_base.cc:1034] running on GCE node
W20260619 22:21:14.364387 6891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:14.364388 6893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:14.364387 6890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:14.365298 6885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:14.366488 6885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:14.367647 6885 hybrid_clock.cc:648] HybridClock initialized: now 1781907674367633 us; error 33 us; skew 500 ppm
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:14.370632 6885 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:14.371644 6885 webserver.cc:460] Webserver started at http://127.1.74.190:44265/ using document root <none> and password file <none>
I20260619 22:21:14.372236 6885 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:14.372327 6885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:14.372557 6885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:14.374706 6885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "f64f71c81c0449018f092681fdabf772"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "3dd27fc6843b14d1be0f073f089ba8d0"
server_key_iv: "123663e07db8068db86bfb50e2b5c917"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.375346 6885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "f64f71c81c0449018f092681fdabf772"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "3dd27fc6843b14d1be0f073f089ba8d0"
server_key_iv: "123663e07db8068db86bfb50e2b5c917"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.379266 6885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:14.381798 6900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.383150 6885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:14.383281 6885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "f64f71c81c0449018f092681fdabf772"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "3dd27fc6843b14d1be0f073f089ba8d0"
server_key_iv: "123663e07db8068db86bfb50e2b5c917"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.383396 6885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:14.407526 6885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:14.410912 6885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:14.411150 6885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:14.418409 6885 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40041
I20260619 22:21:14.418490 6952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40041 every 8 connection(s)
I20260619 22:21:14.419494 6885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:14.422396 6953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:14.428174 6953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772: Bootstrap starting.
I20260619 22:21:14.428526 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6885
I20260619 22:21:14.428655 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:14.428910 1322 external_mini_cluster.cc:1428] Setting key 17f855ecae113efb94252d1522b182fa
I20260619 22:21:14.430929 6953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:14.431710 6953 log.cc:824] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:14.433650 6953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772: No bootstrap required, opened a new log
I20260619 22:21:14.437318 6953 raft_consensus.cc:348] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64f71c81c0449018f092681fdabf772" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40041 } }
I20260619 22:21:14.437582 6953 raft_consensus.cc:374] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:14.437657 6953 raft_consensus.cc:729] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f64f71c81c0449018f092681fdabf772, State: Initialized, Role: FOLLOWER
I20260619 22:21:14.438190 6953 consensus_queue.cc:260] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64f71c81c0449018f092681fdabf772" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40041 } }
I20260619 22:21:14.438346 6953 raft_consensus.cc:388] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:14.438413 6953 raft_consensus.cc:482] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:14.438516 6953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:14.439736 6953 raft_consensus.cc:504] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64f71c81c0449018f092681fdabf772" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40041 } }
I20260619 22:21:14.440091 6953 leader_election.cc:302] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f64f71c81c0449018f092681fdabf772; no voters:
I20260619 22:21:14.440438 6953 leader_election.cc:288] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:14.440513 6958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:14.440721 6958 raft_consensus.cc:686] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [term 1 LEADER]: Becoming Leader. State: Replica: f64f71c81c0449018f092681fdabf772, State: Running, Role: LEADER
I20260619 22:21:14.441078 6958 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64f71c81c0449018f092681fdabf772" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40041 } }
I20260619 22:21:14.441802 6953 sys_catalog.cc:564] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:14.442338 6959 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f64f71c81c0449018f092681fdabf772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64f71c81c0449018f092681fdabf772" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40041 } } }
I20260619 22:21:14.442503 6959 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:14.442830 6960 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f64f71c81c0449018f092681fdabf772. Latest consensus state: current_term: 1 leader_uuid: "f64f71c81c0449018f092681fdabf772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64f71c81c0449018f092681fdabf772" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40041 } } }
I20260619 22:21:14.443051 6960 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:14.442991 6964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:14.445911 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:14.430434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.1.74.190:40041)
0619 22:21:14.430954 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:14.430971 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:14.431036 (+ 65us) server_negotiation.cc:407] Connection header received
0619 22:21:14.431761 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:14.431805 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:14.432199 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:14.432574 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:14.433471 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.434598 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.435619 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.436170 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.436380 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:14.439516 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:14.439521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:14.439548 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:14.439579 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:14.441658 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:14.442236 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:14.442242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:14.442249 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:14.442411 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:14.442835 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:14.442863 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:14.442869 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:14.443461 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:14.443724 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:14.444549 (+ 825us) server_negotiation.cc:299] Negotiation successful
0619 22:21:14.444882 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":163,"threads_started":1}
I20260619 22:21:14.447567 6964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:14.453879 6964 catalog_manager.cc:1347] Generated new cluster ID: 519919d8ec324e2ca38469ba72134458
I20260619 22:21:14.453974 6964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:14.463914 6964 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:14.465052 6964 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:14.474202 6964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f64f71c81c0449018f092681fdabf772: Generated new TSK 0
I20260619 22:21:14.475071 6964 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:14.505769 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40041
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:14.639708 6979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:14.640012 6979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:14.640058 6979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:14.644464 6979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:14.644546 6979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:14.644608 6979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:14.650151 6979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:14.651312 6979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:14.652310 6979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:14.662207 6985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:14.662200 6987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:14.662209 6984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:14.662460 6979 server_base.cc:1034] running on GCE node
I20260619 22:21:14.663650 6979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:14.664391 6979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:14.665576 6979 hybrid_clock.cc:648] HybridClock initialized: now 1781907674665559 us; error 33 us; skew 500 ppm
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:14.668475 6979 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:14.669370 6979 webserver.cc:460] Webserver started at http://127.1.74.129:37989/ using document root <none> and password file <none>
I20260619 22:21:14.669925 6979 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:14.669986 6979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:14.670157 6979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:14.672315 6979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "f9f0353c85c44355980220d389e5abdf"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "c3a6b5f84ebd6ea62bbe32fc732bbd9b"
server_key_iv: "3fbd6832291c32780fad11725b79eb5b"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.672865 6979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "f9f0353c85c44355980220d389e5abdf"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "c3a6b5f84ebd6ea62bbe32fc732bbd9b"
server_key_iv: "3fbd6832291c32780fad11725b79eb5b"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.676937 6979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:14.679629 6994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.680977 6979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:14.681129 6979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "f9f0353c85c44355980220d389e5abdf"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "c3a6b5f84ebd6ea62bbe32fc732bbd9b"
server_key_iv: "3fbd6832291c32780fad11725b79eb5b"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.681453 6979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:14.718461 6979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:14.721670 6979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:14.721887 6979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:14.722515 6979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:14.723438 6979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:14.723507 6979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.723594 6979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:14.723639 6979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.732419 6979 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:46237
I20260619 22:21:14.732441 7107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:46237 every 8 connection(s)
I20260619 22:21:14.733430 6979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:14.743065 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 6979
I20260619 22:21:14.743208 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:14.743542 1322 external_mini_cluster.cc:1428] Setting key e98c9fd26497448c019418d6590197b1
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:14.748381 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:14.735420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:58497 (local address 127.1.74.190:40041)
0619 22:21:14.735574 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:14.735578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:14.736492 (+ 914us) server_negotiation.cc:407] Connection header received
0619 22:21:14.738040 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:14.738044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:14.738102 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:14.738212 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:14.739564 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.740104 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.741253 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.741530 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.741637 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:14.744485 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:14.744488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:14.744490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:14.744514 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:14.746222 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:14.746808 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:14.746812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:14.746814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:14.746946 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:14.747297 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:14.747301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:14.747303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:14.747484 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:14.747587 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:14.748080 (+ 493us) server_negotiation.cc:299] Negotiation successful
0619 22:21:14.748207 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 22:21:14.749249 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:14.735854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40041 (local address 127.1.74.129:58497)
0619 22:21:14.736322 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:14.736368 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:14.737829 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:14.738394 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:14.738402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:14.738912 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:14.739405 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:14.739418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.740228 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.740232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:14.741123 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:14.741132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.741710 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.741716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:14.741893 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:14.742523 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:14.742540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:14.744328 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:14.746352 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:14.746357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:14.746371 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:14.746704 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:14.747057 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:14.747060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:14.747062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:14.747204 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:14.747585 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:14.747592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:14.747848 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 22:21:14.748041 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 22:21:14.748389 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":121,"threads_started":1}
I20260619 22:21:14.750595 7108 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40041
I20260619 22:21:14.750896 7108 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:14.751406 7108 heartbeater.cc:511] Master 127.1.74.190:40041 requested a full tablet report, sending...
I20260619 22:21:14.752966 6917 ts_manager.cc:194] Registered new tserver with Master: f9f0353c85c44355980220d389e5abdf (127.1.74.129:46237)
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:14.754148 6917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:58497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:21:14.773556 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40041
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:14.888195 7113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:14.888553 7113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:14.888649 7113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:14.892958 7113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:14.893083 7113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:14.893189 7113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:14.898813 7113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:14.900035 7113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:14.901113 7113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:14.911727 7119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:14.911727 7118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:14.911918 7113 server_base.cc:1034] running on GCE node
W20260619 22:21:14.911736 7121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:14.912438 7113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:14.913092 7113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:14.914269 7113 hybrid_clock.cc:648] HybridClock initialized: now 1781907674914233 us; error 48 us; skew 500 ppm
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:14.917083 7113 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:14.917951 7113 webserver.cc:460] Webserver started at http://127.1.74.130:43295/ using document root <none> and password file <none>
I20260619 22:21:14.918558 7113 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:14.918658 7113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:14.918937 7113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:14.921168 7113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "c71200849ba14797ac3d5223e41a561a"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "919f96c9224be1d1094f26eeaadba0e6"
server_key_iv: "3da1e2f21db268e532284b17735e0f15"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.921734 7113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "c71200849ba14797ac3d5223e41a561a"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "919f96c9224be1d1094f26eeaadba0e6"
server_key_iv: "3da1e2f21db268e532284b17735e0f15"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.925523 7113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:14.928051 7128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.929282 7113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:14.929420 7113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "c71200849ba14797ac3d5223e41a561a"
format_stamp: "Formatted at 2026-06-19 22:21:14 on dist-test-slave-k84h"
server_key: "919f96c9224be1d1094f26eeaadba0e6"
server_key_iv: "3da1e2f21db268e532284b17735e0f15"
server_key_version: "encryptionkey@0"
I20260619 22:21:14.929536 7113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:14.944420 7113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:14.947562 7113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:14.947784 7113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:14.948400 7113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:14.949357 7113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:14.949426 7113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.949497 7113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:14.949543 7113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:14.959039 7113 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:45715
I20260619 22:21:14.959071 7241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:45715 every 8 connection(s)
I20260619 22:21:14.960021 7113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:14.960873 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7113
I20260619 22:21:14.960991 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:14.961308 1322 external_mini_cluster.cc:1428] Setting key bbb5bce30861cbfb23650cc480f18acc
Jun 19 22:21:14 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907674, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:21:14.976758 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:14.962033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:43981 (local address 127.1.74.190:40041)
0619 22:21:14.962193 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:14.962196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:14.963314 (+ 1118us) server_negotiation.cc:407] Connection header received
0619 22:21:14.965049 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:14.965052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:14.965101 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:14.965169 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:14.966487 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.967112 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.969021 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.969243 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.969323 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:14.972666 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:14.972669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:14.972671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:14.972701 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:14.974333 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:14.975029 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:14.975032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:14.975033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:14.975148 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:14.975553 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:14.975557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:14.975558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:14.975794 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:14.975911 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:14.976450 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 22:21:14.976586 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 22:21:14.977977 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:14.962602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40041 (local address 127.1.74.130:43981)
0619 22:21:14.963145 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:14.963200 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:14.964771 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:14.965414 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:14.965425 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:14.965836 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:14.966340 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:14.966354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.967240 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.967247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:14.968860 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:14.968877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:14.969417 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:14.969422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:14.969680 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:14.970377 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:14.970400 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:14.972410 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:14.974470 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:14.974477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:14.974519 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:14.974809 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:14.975264 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:14.975268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:14.975272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:14.975442 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:14.975905 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:14.975913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:14.976195 (+ 282us) client_negotiation.cc:769] Sending connection context
0619 22:21:14.976390 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 22:21:14.976767 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":163,"threads_started":1}
I20260619 22:21:14.979298 7242 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40041
I20260619 22:21:14.979564 7242 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:14.980137 7242 heartbeater.cc:511] Master 127.1.74.190:40041 requested a full tablet report, sending...
I20260619 22:21:14.981356 6917 ts_manager.cc:194] Registered new tserver with Master: c71200849ba14797ac3d5223e41a561a (127.1.74.130:45715)
I20260619 22:21:14.982278 6917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:43981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:14.993840 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40041
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:15.105736 7247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:15.106037 7247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:15.106082 7247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:15.110512 7247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:15.110589 7247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:15.110694 7247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:15.116187 7247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:15.117304 7247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:15.118360 7247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:15.128252 7253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:15.128252 7252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:15.128254 7255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:15.128546 7247 server_base.cc:1034] running on GCE node
I20260619 22:21:15.129107 7247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:15.129894 7247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:15.131120 7247 hybrid_clock.cc:648] HybridClock initialized: now 1781907675131093 us; error 41 us; skew 500 ppm
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.134443 7247 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:15.135495 7247 webserver.cc:460] Webserver started at http://127.1.74.131:44177/ using document root <none> and password file <none>
I20260619 22:21:15.136097 7247 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:15.136186 7247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:15.136418 7247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:15.138689 7247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "02a3f1dbe96e4b38b0e09c84b14bf8c7"
format_stamp: "Formatted at 2026-06-19 22:21:15 on dist-test-slave-k84h"
server_key: "12afdfa95692244db1ebe1fb5f25054c"
server_key_iv: "ba86ae0336511b7cac38cb049f9d2eae"
server_key_version: "encryptionkey@0"
I20260619 22:21:15.139351 7247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "02a3f1dbe96e4b38b0e09c84b14bf8c7"
format_stamp: "Formatted at 2026-06-19 22:21:15 on dist-test-slave-k84h"
server_key: "12afdfa95692244db1ebe1fb5f25054c"
server_key_iv: "ba86ae0336511b7cac38cb049f9d2eae"
server_key_version: "encryptionkey@0"
I20260619 22:21:15.143493 7247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:15.145996 7262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:15.147347 7247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:15.147480 7247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "02a3f1dbe96e4b38b0e09c84b14bf8c7"
format_stamp: "Formatted at 2026-06-19 22:21:15 on dist-test-slave-k84h"
server_key: "12afdfa95692244db1ebe1fb5f25054c"
server_key_iv: "ba86ae0336511b7cac38cb049f9d2eae"
server_key_version: "encryptionkey@0"
I20260619 22:21:15.147590 7247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:15.164161 7247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:15.167294 7247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:15.167495 7247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:15.168103 7247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:15.168937 7247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:15.169014 7247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:15.169082 7247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:15.169128 7247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:15.178056 7247 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:33015
I20260619 22:21:15.178081 7375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:33015 every 8 connection(s)
I20260619 22:21:15.179004 7247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:15.181214 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7247
I20260619 22:21:15.181337 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:15.181663 1322 external_mini_cluster.cc:1428] Setting key 3885f5837cb80e679bc1cbd1750f2f66
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:15.193493 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:15.180897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:52533 (local address 127.1.74.190:40041)
0619 22:21:15.181036 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.181039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.182262 (+ 1223us) server_negotiation.cc:407] Connection header received
0619 22:21:15.184135 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.184138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.184181 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.184250 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.185368 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.186018 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.187229 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.187457 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.187562 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:15.190047 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:15.190050 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:15.190052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:15.190078 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:15.191573 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.192053 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.192055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.192057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.192132 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.192442 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.192445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.192446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.192658 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:15.192749 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:15.193259 (+ 510us) server_negotiation.cc:299] Negotiation successful
0619 22:21:15.193368 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:21:15.194317 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:15.181495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40041 (local address 127.1.74.131:52533)
0619 22:21:15.182087 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:15.182156 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:15.183916 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:15.184386 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:15.184393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:15.184770 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:15.185237 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:15.185249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.186119 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.186121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:15.187092 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:15.187119 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.187615 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.187619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:15.187793 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:15.188339 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:15.188356 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:15.189874 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:15.191673 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:15.191678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:15.191690 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:15.191956 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:15.192210 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:15.192212 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:15.192213 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:15.192337 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:15.192793 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:15.192799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:15.193049 (+ 250us) client_negotiation.cc:769] Sending connection context
0619 22:21:15.193218 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 22:21:15.193478 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":141,"threads_started":1}
I20260619 22:21:15.195472 7376 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40041
I20260619 22:21:15.195729 7376 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:15.196265 7376 heartbeater.cc:511] Master 127.1.74.190:40041 requested a full tablet report, sending...
I20260619 22:21:15.197317 6917 ts_manager.cc:194] Registered new tserver with Master: 02a3f1dbe96e4b38b0e09c84b14bf8c7 (127.1.74.131:33015)
I20260619 22:21:15.197876 6917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:52533
I20260619 22:21:15.207702 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/test-admin.keytab.
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.232658 1322 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 22:21:15.235872 7383 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:15.234175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.1.74.129:46237)
0619 22:21:15.234488 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.234504 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.234532 (+ 28us) server_negotiation.cc:407] Connection header received
0619 22:21:15.234666 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.234672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.234916 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.235041 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.235670 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44078: BlockingRecv error: recv got EOF from 127.0.0.1:44078 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
W20260619 22:21:15.235955 1322 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc' of type FILE
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.237138 1322 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.238314 1322 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:21:15.249972 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:15.239421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.1.74.129:46237)
0619 22:21:15.239565 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.239569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.239750 (+ 181us) server_negotiation.cc:407] Connection header received
0619 22:21:15.239832 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.239838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.239890 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.239987 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.240465 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.241876 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.242883 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.243537 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.243674 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:15.246334 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:15.246343 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:15.246355 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:15.246379 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:15.248074 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.248568 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.248572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.248574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.248645 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.248930 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.248932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.248933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.249221 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:15.249370 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:15.249626 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 22:21:15.249743 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 22:21:15.254170 7383 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:15.253165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44092 (local address 127.1.74.129:46237)
0619 22:21:15.253293 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.253297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.253403 (+ 106us) server_negotiation.cc:407] Connection header received
0619 22:21:15.253554 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.253558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.253598 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.253683 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.254063 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44092: BlockingRecv error: recv got EOF from 127.0.0.1:44092 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 22:21:15.254266 1322 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc' of type FILE
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.255501 1322 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.256472 1322 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:21:15.266031 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:15.257303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.1.74.129:46237)
0619 22:21:15.257414 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.257420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.257517 (+ 97us) server_negotiation.cc:407] Connection header received
0619 22:21:15.257610 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.257612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.257643 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.257705 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.258315 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.258833 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.260017 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.260237 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.260313 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:15.262769 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:15.262775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:15.262778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:15.262808 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:15.264411 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.264797 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.264803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.264806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.264894 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.265132 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.265137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.265139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.265351 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:15.265485 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:15.265683 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 22:21:15.265814 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 22:21:15.268765 7383 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:15.267869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.1.74.129:46237)
0619 22:21:15.268010 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.268016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.268074 (+ 58us) server_negotiation.cc:407] Connection header received
0619 22:21:15.268165 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.268170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.268216 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.268307 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.268651 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44108: BlockingRecv error: recv got EOF from 127.0.0.1:44108 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 22:21:15.268776 1322 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781907643212257-1322-0/krb5kdc/krb5cc' of type FILE
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.269914 1322 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:15.271121 1322 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[6872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:21:15.284710 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:15.272058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.1.74.129:46237)
0619 22:21:15.272166 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:15.272170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:15.272317 (+ 147us) server_negotiation.cc:407] Connection header received
0619 22:21:15.272419 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:15.272421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:15.272458 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:15.272528 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:15.273205 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.273685 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.274938 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:15.275160 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:15.275243 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:15.281972 (+ 6729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:15.281978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:15.281983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:15.282020 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:15.283442 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.283810 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.283812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.283813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.283878 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:15.284084 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:15.284086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:15.284088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:15.284236 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:15.284328 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:15.284508 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 22:21:15.284593 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:21:15.285730 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6979
I20260619 22:21:15.291285 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7113
I20260619 22:21:15.296869 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7247
I20260619 22:21:15.301924 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 6885
2026-06-19T22:21:15Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2241 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[7396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[7396](info): set up 2 sockets
Jun 19 22:21:15 dist-test-slave-k84h krb5kdc[7396](info): commencing operation
krb5kdc: starting...
W20260619 22:21:16.343542 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:16 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab.
Entry for principal oryx/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab.
I20260619 22:21:16.468299 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39085
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab
--principal=oryx/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:16.578572 7409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:16.578925 7409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:16.579032 7409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:16.585814 7409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:16.585911 7409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:16.585949 7409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:16.586004 7409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:16.586053 7409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:16.593132 7409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab
--principal=oryx/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:16.594214 7409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:16.595300 7409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:16.604141 7415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:16.604141 7414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:16.604141 7417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:16.604619 7409 server_base.cc:1034] running on GCE node
I20260619 22:21:16.605270 7409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:16.606521 7409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:16.607743 7409 hybrid_clock.cc:648] HybridClock initialized: now 1781907676607713 us; error 54 us; skew 500 ppm
Jun 19 22:21:16 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:16.610976 7409 init.cc:377] Logged in from keytab as oryx/127.1.74.190@KRBTEST.COM (short username oryx)
I20260619 22:21:16.612007 7409 webserver.cc:460] Webserver started at http://127.1.74.190:44429/ using document root <none> and password file <none>
I20260619 22:21:16.612670 7409 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:16.612772 7409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:16.613029 7409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:16.615420 7409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "c3b1bc7f83df4b2ea1f56bc784bed180"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "5e49399d867473c27f9eb94a63bcc667"
server_key_iv: "62c7372dcce305136df668a1a13fa635"
server_key_version: "encryptionkey@0"
I20260619 22:21:16.616058 7409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "c3b1bc7f83df4b2ea1f56bc784bed180"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "5e49399d867473c27f9eb94a63bcc667"
server_key_iv: "62c7372dcce305136df668a1a13fa635"
server_key_version: "encryptionkey@0"
I20260619 22:21:16.620138 7409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:16.622920 7424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:16.624320 7409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:16.624481 7409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "c3b1bc7f83df4b2ea1f56bc784bed180"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "5e49399d867473c27f9eb94a63bcc667"
server_key_iv: "62c7372dcce305136df668a1a13fa635"
server_key_version: "encryptionkey@0"
I20260619 22:21:16.624612 7409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:16.648926 7409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:16.652360 7409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:16.652606 7409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:16.660512 7409 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39085
I20260619 22:21:16.660807 7476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39085 every 8 connection(s)
I20260619 22:21:16.661710 7409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:16.664716 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:16.665122 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7409
I20260619 22:21:16.665271 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:16.665567 1322 external_mini_cluster.cc:1428] Setting key 746313b7ac5e59e855b493604996ec4d
I20260619 22:21:16.672475 7477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Bootstrap starting.
Jun 19 22:21:16 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:16.675393 7477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:16.676486 7477 log.cc:824] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:16.678799 7477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: No bootstrap required, opened a new log
I20260619 22:21:16.680470 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:16.666942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.1.74.190:39085)
0619 22:21:16.667438 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:16.667454 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:16.667511 (+ 57us) server_negotiation.cc:407] Connection header received
0619 22:21:16.668177 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:16.668197 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:16.668573 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:16.668940 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:16.669788 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:16.670825 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:16.671781 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:16.672334 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:16.672522 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:16.675332 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:16.675338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:16.675353 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:16.675386 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:16.677539 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:16.678010 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:16.678020 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:16.678025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:16.678138 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:16.678429 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:16.678432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:16.678434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:16.678887 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:16.679088 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:16.679389 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 22:21:16.679694 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":165,"threads_started":1}
I20260619 22:21:16.683499 7477 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } }
I20260619 22:21:16.683771 7477 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:16.683862 7477 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b1bc7f83df4b2ea1f56bc784bed180, State: Initialized, Role: FOLLOWER
I20260619 22:21:16.684317 7477 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } }
I20260619 22:21:16.684485 7477 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:16.684571 7477 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:16.684687 7477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:16.685750 7477 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } }
I20260619 22:21:16.686084 7477 leader_election.cc:302] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b1bc7f83df4b2ea1f56bc784bed180; no voters:
I20260619 22:21:16.686393 7477 leader_election.cc:288] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:16.686579 7482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:16.686832 7482 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 LEADER]: Becoming Leader. State: Replica: c3b1bc7f83df4b2ea1f56bc784bed180, State: Running, Role: LEADER
I20260619 22:21:16.687268 7482 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } }
I20260619 22:21:16.687851 7477 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:16.689182 7483 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } }
I20260619 22:21:16.689190 7484 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b1bc7f83df4b2ea1f56bc784bed180. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } }
I20260619 22:21:16.689361 7484 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:16.689359 7483 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:16.690515 7491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:16.693881 7491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:16.700065 7491 catalog_manager.cc:1347] Generated new cluster ID: 16beac010ee74b349c1a6c813c8158e9
I20260619 22:21:16.700184 7491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:16.729140 7491 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:16.730448 7491 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:16.740262 7491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Generated new TSK 0
I20260619 22:21:16.741044 7491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab.
Entry for principal oryx/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab.
I20260619 22:21:16.777441 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39085
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:16.903157 7503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:16.903424 7503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:16.903469 7503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:16.907452 7503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:16.907517 7503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:16.907577 7503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:16.912788 7503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:16.913815 7503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:16.914800 7503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:16.924707 7508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:16.924707 7509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:16.924770 7511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:16.925092 7503 server_base.cc:1034] running on GCE node
I20260619 22:21:16.925525 7503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:16.926169 7503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:16.927392 7503 hybrid_clock.cc:648] HybridClock initialized: now 1781907676927368 us; error 39 us; skew 500 ppm
Jun 19 22:21:16 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:16.930635 7503 init.cc:377] Logged in from keytab as oryx/127.1.74.129@KRBTEST.COM (short username oryx)
I20260619 22:21:16.931646 7503 webserver.cc:460] Webserver started at http://127.1.74.129:46443/ using document root <none> and password file <none>
I20260619 22:21:16.932209 7503 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:16.932297 7503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:16.932524 7503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:16.934584 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "0362f0c1fc754d54be8d60b73956df39"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "b0329130b9646193e356c8c163ef7184"
server_key_iv: "4fd14868020026927703e5eddc6a57c9"
server_key_version: "encryptionkey@0"
I20260619 22:21:16.935166 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "0362f0c1fc754d54be8d60b73956df39"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "b0329130b9646193e356c8c163ef7184"
server_key_iv: "4fd14868020026927703e5eddc6a57c9"
server_key_version: "encryptionkey@0"
I20260619 22:21:16.938910 7503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:21:16.941569 7518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:16.942984 7503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:16.943138 7503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "0362f0c1fc754d54be8d60b73956df39"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "b0329130b9646193e356c8c163ef7184"
server_key_iv: "4fd14868020026927703e5eddc6a57c9"
server_key_version: "encryptionkey@0"
I20260619 22:21:16.943262 7503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:16.960654 7503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:16.964638 7503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:16.964869 7503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:16.965564 7503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:16.966428 7503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:16.966483 7503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:16.966529 7503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:16.966547 7503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:16.976570 7503 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:40109
I20260619 22:21:16.976589 7631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:40109 every 8 connection(s)
I20260619 22:21:16.977576 7503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:16.984819 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7503
I20260619 22:21:16.984925 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:16.985208 1322 external_mini_cluster.cc:1428] Setting key 9a18bb1a934e4bb9c97ce2eb49c55bae
Jun 19 22:21:16 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:16.993304 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:16.979528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:50257 (local address 127.1.74.190:39085)
0619 22:21:16.979671 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:16.979675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:16.980521 (+ 846us) server_negotiation.cc:407] Connection header received
0619 22:21:16.981907 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:16.981911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:16.981970 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:16.982054 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:16.983392 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:16.983945 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:16.985213 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:16.985496 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:16.985623 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:16.988755 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:16.988761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:16.988765 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:16.988798 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:16.990711 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:16.991337 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:16.991343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:16.991348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:16.991479 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:16.991869 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:16.991873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:16.991875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:16.992167 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:16.992292 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:16.992871 (+ 579us) server_negotiation.cc:299] Negotiation successful
0619 22:21:16.993041 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 22:21:16.994032 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:16.979895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:50257)
0619 22:21:16.980364 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:16.980409 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:16.981685 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:16.982239 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:16.982246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:16.982626 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:16.983233 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:16.983248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:16.984080 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:16.984088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:16.985087 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:16.985097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:16.985668 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:16.985672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:16.985858 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:16.986508 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:16.986525 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:16.988456 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:16.990834 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:16.990873 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:16.990896 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:16.991198 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:16.991597 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:16.991600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:16.991601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:16.991754 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:16.992312 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:16.992318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:16.992594 (+ 276us) client_negotiation.cc:769] Sending connection context
0619 22:21:16.992768 (+ 174us) client_negotiation.cc:240] Negotiation successful
0619 22:21:16.993073 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260619 22:21:16.995381 7632 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39085
WARNING: no policy specified for oryx/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:16.995635 7632 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:16.996142 7632 heartbeater.cc:511] Master 127.1.74.190:39085 requested a full tablet report, sending...
I20260619 22:21:16.997684 7441 ts_manager.cc:194] Registered new tserver with Master: 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.130@KRBTEST.COM" created.
I20260619 22:21:16.999146 7441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:50257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab.
Entry for principal oryx/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab.
I20260619 22:21:17.014668 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39085
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:17.131338 7637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:17.131637 7637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:17.131685 7637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:17.135998 7637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:17.136073 7637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:17.136166 7637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:17.142735 7637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:17.144177 7637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:17.145547 7637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:17.155819 7645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.155814 7642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.155875 7643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.156203 7637 server_base.cc:1034] running on GCE node
I20260619 22:21:17.156574 7637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:17.157243 7637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:17.158447 7637 hybrid_clock.cc:648] HybridClock initialized: now 1781907677158404 us; error 54 us; skew 500 ppm
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:17.161412 7637 init.cc:377] Logged in from keytab as oryx/127.1.74.130@KRBTEST.COM (short username oryx)
I20260619 22:21:17.162324 7637 webserver.cc:460] Webserver started at http://127.1.74.130:37889/ using document root <none> and password file <none>
I20260619 22:21:17.162964 7637 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:17.163059 7637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:17.163298 7637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:17.165405 7637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "247dc117e29441ee9cb12818c64ede15"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "e305eeb6d206bb1fda2410432ca2d66a"
server_key_iv: "d44f90f8653f69b3275273826aa368de"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.165988 7637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "247dc117e29441ee9cb12818c64ede15"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "e305eeb6d206bb1fda2410432ca2d66a"
server_key_iv: "d44f90f8653f69b3275273826aa368de"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.169896 7637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:21:17.172552 7652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.173848 7637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:17.173985 7637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "247dc117e29441ee9cb12818c64ede15"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "e305eeb6d206bb1fda2410432ca2d66a"
server_key_iv: "d44f90f8653f69b3275273826aa368de"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.174113 7637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:17.192023 7637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:17.195447 7637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:17.195669 7637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:17.196302 7637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:17.197160 7637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:17.197232 7637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.197314 7637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:17.197366 7637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.207490 7637 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:36505
I20260619 22:21:17.207521 7765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:36505 every 8 connection(s)
I20260619 22:21:17.208451 7637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:17.211377 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7637
I20260619 22:21:17.211508 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:17.211854 1322 external_mini_cluster.cc:1428] Setting key c92fc49cf82c9135f00e3a690688fc40
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.131@KRBTEST.COM" created.
I20260619 22:21:17.224064 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.210540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:56639 (local address 127.1.74.190:39085)
0619 22:21:17.210698 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.210702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.211683 (+ 981us) server_negotiation.cc:407] Connection header received
0619 22:21:17.213529 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.213534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.213586 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.213694 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.215338 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.215882 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.217199 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.217435 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.217551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.220357 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.220361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.220363 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.220386 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.221881 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.222413 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.222416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.222417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.222509 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.222895 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.222905 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.222906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.223099 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.223206 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.223754 (+ 548us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.223878 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 22:21:17.224834 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.210995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:56639)
0619 22:21:17.211501 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.211558 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.213064 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.214037 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.214061 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.214611 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.215190 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.215205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.216153 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.216157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.217073 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.217082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.217612 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.217616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.217780 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.218437 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.218453 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.220175 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.221992 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.221997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.222010 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.222312 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.222621 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.222625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.222627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.222770 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.223204 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.223211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.223458 (+ 247us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.223660 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.223992 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":136,"threads_started":1}
I20260619 22:21:17.226121 7766 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39085
I20260619 22:21:17.226382 7766 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:17.227058 7766 heartbeater.cc:511] Master 127.1.74.190:39085 requested a full tablet report, sending...
I20260619 22:21:17.228399 7441 ts_manager.cc:194] Registered new tserver with Master: 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505)
I20260619 22:21:17.229053 7441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:56639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab.
Entry for principal oryx/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab.
I20260619 22:21:17.239739 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39085
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:17.353394 7771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:17.353688 7771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:17.353765 7771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:17.357874 7771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:17.357961 7771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:17.358057 7771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:17.363333 7771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:17.364382 7771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:17.365383 7771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:17.374933 7779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.375070 7771 server_base.cc:1034] running on GCE node
W20260619 22:21:17.374956 7776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.375276 7777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.375727 7771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:17.376377 7771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:17.377579 7771 hybrid_clock.cc:648] HybridClock initialized: now 1781907677377568 us; error 29 us; skew 500 ppm
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:17.380543 7771 init.cc:377] Logged in from keytab as oryx/127.1.74.131@KRBTEST.COM (short username oryx)
I20260619 22:21:17.381503 7771 webserver.cc:460] Webserver started at http://127.1.74.131:38987/ using document root <none> and password file <none>
I20260619 22:21:17.382093 7771 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:17.382200 7771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:17.382421 7771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:17.384586 7771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "e6fc676f2fb7487c93e6f8f85d30de64"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "a16a20b2ed03220c5d94e3eec898803f"
server_key_iv: "e0852d44bd8fc48ec740f43fe71d621d"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.385159 7771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "e6fc676f2fb7487c93e6f8f85d30de64"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "a16a20b2ed03220c5d94e3eec898803f"
server_key_iv: "e0852d44bd8fc48ec740f43fe71d621d"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.389348 7771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260619 22:21:17.392113 7786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.393421 7771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:17.393597 7771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "e6fc676f2fb7487c93e6f8f85d30de64"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "a16a20b2ed03220c5d94e3eec898803f"
server_key_iv: "e0852d44bd8fc48ec740f43fe71d621d"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.393723 7771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:17.408958 7771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:17.412060 7771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:17.412262 7771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:17.412860 7771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:17.413690 7771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:17.413760 7771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.413832 7771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:17.413884 7771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.422469 7771 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:33087
I20260619 22:21:17.422541 7899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:33087 every 8 connection(s)
I20260619 22:21:17.423539 7771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:17.426383 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7771
I20260619 22:21:17.426560 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:17.426913 1322 external_mini_cluster.cc:1428] Setting key 8b400a98c729082677bec9c4e2b2aa15
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:17.437994 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.425394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:33027 (local address 127.1.74.190:39085)
0619 22:21:17.425569 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.425573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.426488 (+ 915us) server_negotiation.cc:407] Connection header received
0619 22:21:17.428223 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.428227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.428287 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.428369 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.429702 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.430259 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.431487 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.431715 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.431827 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.434418 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.434427 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.434429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.434453 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.435930 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.436452 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.436457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.436459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.436548 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.436890 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.436894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.436896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.437083 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.437189 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.437703 (+ 514us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.437828 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:17.438916 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.425810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:33027)
0619 22:21:17.426311 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.426375 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.427946 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.428579 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.428586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.428971 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.429562 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.429576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.430385 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.430389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.431360 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.431369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.431865 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.431869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.432054 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.432702 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.432718 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.434235 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.436056 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.436062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.436079 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.436344 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.436644 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.436648 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.436650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.436791 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.437189 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.437197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.437459 (+ 262us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.437654 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.437991 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":106,"threads_started":1}
I20260619 22:21:17.440244 7900 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39085
I20260619 22:21:17.440524 7900 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:17.441179 7900 heartbeater.cc:511] Master 127.1.74.190:39085 requested a full tablet report, sending...
I20260619 22:21:17.442251 7441 ts_manager.cc:194] Registered new tserver with Master: e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087)
I20260619 22:21:17.442914 7441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:33027
I20260619 22:21:17.452903 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.189.keytab.
Entry for principal oryx/127.1.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.189.keytab.
I20260619 22:21:17.479103 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:34295
--webserver_interface=127.1.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.189.keytab
--principal=oryx/127.1.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39085,127.1.74.189:34295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:17.589890 7905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:17.590204 7905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:17.590294 7905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:17.594408 7905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:17.594511 7905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:17.594548 7905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:17.594586 7905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:17.594622 7905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:17.600205 7905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39085,127.1.74.189:34295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.189.keytab
--principal=oryx/127.1.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:34295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
W20260619 22:21:17.600472 7905 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.74.190:39085,127.1.74.189:34295'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 22:21:17.601387 7905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:17.602421 7905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:17.610725 7910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.610728 7913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.610725 7911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.611033 7905 server_base.cc:1034] running on GCE node
I20260619 22:21:17.611649 7905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:17.612802 7905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:17.613997 7905 hybrid_clock.cc:648] HybridClock initialized: now 1781907677613955 us; error 51 us; skew 500 ppm
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:17.617329 7905 init.cc:377] Logged in from keytab as oryx/127.1.74.189@KRBTEST.COM (short username oryx)
I20260619 22:21:17.618203 7905 webserver.cc:460] Webserver started at http://127.1.74.189:42065/ using document root <none> and password file <none>
I20260619 22:21:17.618963 7905 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:17.619079 7905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:17.619304 7905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:17.621531 7905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/instance:
uuid: "372fc71d57c940e3bacacd31cf83db4f"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "0768f672b457a076d2cfe0b27e2f4b81"
server_key_iv: "79b168251acff9c6277e26aa245dce90"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.622059 7905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal/instance:
uuid: "372fc71d57c940e3bacacd31cf83db4f"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "0768f672b457a076d2cfe0b27e2f4b81"
server_key_iv: "79b168251acff9c6277e26aa245dce90"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.625921 7905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 22:21:17.628594 7920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.629998 7905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:17.630101 7905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
uuid: "372fc71d57c940e3bacacd31cf83db4f"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "0768f672b457a076d2cfe0b27e2f4b81"
server_key_iv: "79b168251acff9c6277e26aa245dce90"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.630196 7905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:17.656870 7905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:17.660424 7905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:17.660641 7905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:17.668303 7905 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.189:34295
I20260619 22:21:17.668604 7972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.189:34295 every 8 connection(s)
I20260619 22:21:17.669406 7905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
I20260619 22:21:17.672334 7973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:17.675462 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 7905
I20260619 22:21:17.676940 7973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.189@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
I20260619 22:21:17.690984 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.678704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.1.74.190:39085)
0619 22:21:17.678833 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.678837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.679233 (+ 396us) server_negotiation.cc:407] Connection header received
0619 22:21:17.679484 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.679487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.679551 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.679633 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.680495 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.681309 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.682818 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.683173 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.683292 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.686256 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.686261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.686268 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.686306 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.688019 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.688690 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.688693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.688694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.688775 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.689619 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.689623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.689625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.690052 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.690133 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.690620 (+ 487us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.690770 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:21:17.692085 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.678633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55204)
0619 22:21:17.679089 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.679144 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.679353 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.679719 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.679725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.680200 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.680357 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.680367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.681446 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.681448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.682698 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.682706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.683378 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.683383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.683496 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.684207 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.684229 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.686006 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.688142 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.688149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.688163 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.688504 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.689333 (+ 829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.689335 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.689337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.689498 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.690151 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.690158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.690342 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.690554 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.690946 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":24,"spinlock_wait_cycles":5888,"thread_start_us":88,"threads_started":1}
I20260619 22:21:17.692093 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.677076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.1.74.189:34295)
0619 22:21:17.677498 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.677518 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.677555 (+ 37us) server_negotiation.cc:407] Connection header received
0619 22:21:17.678205 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.678246 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.678669 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.679103 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.680077 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.681447 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.682656 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.683280 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.683439 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.686716 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.686724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.686740 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.686771 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.688817 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.689326 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.689332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.689334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.689498 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.689836 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.689839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.689840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.690342 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.690524 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.691207 (+ 683us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.691336 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":270,"spinlock_wait_cycles":9984,"thread_start_us":132,"threads_started":1}
I20260619 22:21:17.693732 7973 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.189@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
I20260619 22:21:17.701946 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.694285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:41992)
0619 22:21:17.694402 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.694419 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.694509 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.694838 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.694884 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.695042 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.695146 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.695151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.695841 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.695843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.696360 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.696367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.697225 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.697227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.697303 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.697782 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.697796 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.699160 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.700614 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.700617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.700619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.700803 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.701025 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.701027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.701028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.701098 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.701536 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.701538 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.701627 (+ 89us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.701703 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.701817 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 22:21:17.702042 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.694324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.1.74.189:34295)
0619 22:21:17.694426 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.694430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.694465 (+ 35us) server_negotiation.cc:407] Connection header received
0619 22:21:17.694707 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.694709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.694744 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.694809 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.695238 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.695750 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.696453 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.697124 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.697209 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.699276 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.699279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.699281 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.699302 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.700485 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.700882 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.700884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.700885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.700952 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.701211 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.701214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.701215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.701416 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.701536 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.701751 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.701894 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":27}
I20260619 22:21:17.705065 7973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Bootstrap starting.
I20260619 22:21:17.707405 7973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:17.708144 7973 log.cc:824] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:17.710038 7973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: No bootstrap required, opened a new log
I20260619 22:21:17.713057 7973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } }
I20260619 22:21:17.713299 7973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:17.713378 7973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372fc71d57c940e3bacacd31cf83db4f, State: Initialized, Role: FOLLOWER
I20260619 22:21:17.713824 7973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } }
I20260619 22:21:17.714298 7979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } } }
I20260619 22:21:17.714409 7979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:17.715099 7973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: configured and running, proceeding with master startup.
W20260619 22:21:17.719552 7990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:21:17.719648 7990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:21:17.721750 7905 master_runner.cc:428] Detected that this master 372fc71d57c940e3bacacd31cf83db4f is joining an existing cluster
I20260619 22:21:17.721819 7905 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.74.189:34295
I20260619 22:21:17.732522 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.724328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55208)
0619 22:21:17.724994 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.725006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.725116 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.725402 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.725405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.725603 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.725737 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.725742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.726533 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.726539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.727329 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.727338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.727904 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.727908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.728000 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.728525 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.728540 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.729002 (+ 462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.730819 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.730824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.730826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.731153 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.731447 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.731449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.731452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.731539 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.731919 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.731922 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.732055 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.732167 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.732316 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":438,"thread_start_us":135,"threads_started":1}
I20260619 22:21:17.732509 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.724541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.1.74.190:39085)
0619 22:21:17.724666 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.724670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.725081 (+ 411us) server_negotiation.cc:407] Connection header received
0619 22:21:17.725215 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.725218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.725264 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.725334 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.725857 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.726416 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.727449 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.727764 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.727867 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.729154 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.729159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.729164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.729196 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.730709 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.731255 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.731259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.731261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.731344 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.731631 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.731635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.731637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.731826 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.731932 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.732230 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.732347 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:21:17.733517 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.724328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42008)
0619 22:21:17.724695 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.724710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.724816 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.725131 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.725134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.725366 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.725501 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.725508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.726550 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.726553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.727519 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.727529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.728060 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.728063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.728186 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.728754 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.728778 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.729728 (+ 950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.731983 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.731985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.731988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.732235 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.732488 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.732490 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.732492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.732561 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.732964 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.732968 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.733051 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.733172 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.733310 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":19,"thread_start_us":170,"threads_started":1}
I20260619 22:21:17.733544 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.724400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.1.74.189:34295)
0619 22:21:17.724511 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.724517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.724796 (+ 279us) server_negotiation.cc:407] Connection header received
0619 22:21:17.724918 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.724921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.724959 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.725005 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.725628 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.726430 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.727634 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.727940 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.728060 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.729864 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.729868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.729871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.729894 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.731864 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.732329 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.732332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.732334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.732400 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.732658 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.732661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.732663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.732842 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.732908 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.733234 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.733384 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":31}
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.190@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
I20260619 22:21:17.746958 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.737475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.1.74.189:34295)
0619 22:21:17.737596 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.737600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.737928 (+ 328us) server_negotiation.cc:407] Connection header received
0619 22:21:17.738145 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.738148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.738200 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.738277 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.738925 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.739717 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.740603 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.740814 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.740915 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.743353 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.743362 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.743364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.743384 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.744837 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.745477 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.745481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.745482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.745565 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.745870 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.745873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.745875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.746086 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.746247 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.746655 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.746769 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 22:21:17.747563 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.737436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42016)
0619 22:21:17.737800 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.737853 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.737996 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.738314 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.738318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.738665 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.738769 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.738775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.739839 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.739842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.740471 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.740479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.740921 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.740925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.741062 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.741585 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.741597 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.743212 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.744957 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.744966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.744973 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.745366 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.745661 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.745665 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.745667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.745760 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.746254 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.746261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.746417 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.746588 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.746827 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":115,"threads_started":1}
I20260619 22:21:17.749192 7441 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.74.189:34295: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f"
member_type: NON_VOTER
last_known_addr {
host: "127.1.74.189"
port: 34295
}
attrs {
promote: true
}
}
dest_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180"
cas_config_opid_index: -1
I20260619 22:21:17.749806 7441 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } }
I20260619 22:21:17.751739 8001 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 372fc71d57c940e3bacacd31cf83db4f (127.1.74.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } }
I20260619 22:21:17.752345 7947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:17.753500 8001 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.74.189:34295
I20260619 22:21:17.753746 7947 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Refusing update from remote peer c3b1bc7f83df4b2ea1f56bc784bed180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 22:21:17.753870 8002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } }
I20260619 22:21:17.753983 8002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:17.754297 7905 master.cc:537] Master@127.1.74.189:34295 shutting down...
I20260619 22:21:17.754894 8002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:17.757308 7979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:17.757478 7979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } }
W20260619 22:21:17.758005 7428 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 22:21:17.758275 7979 leader_election.cc:288] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085)
I20260619 22:21:17.758723 7451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "372fc71d57c940e3bacacd31cf83db4f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" is_pre_election: true
I20260619 22:21:17.759404 7923 leader_election.cc:302] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 372fc71d57c940e3bacacd31cf83db4f; no voters: c3b1bc7f83df4b2ea1f56bc784bed180
I20260619 22:21:17.759554 7905 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:17.759644 7979 raft_consensus.cc:2684] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Received pre-election callback for term 2 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260619 22:21:17.759864 7905 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:17.759984 7905 tablet_replica.cc:330] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: stopping tablet replica
WARNING: no policy specified for oryx/127.1.74.188@KRBTEST.COM; defaulting to no policy
I20260619 22:21:17.763808 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.763419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42018)
0619 22:21:17.763632 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.763710 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:34295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260619 22:21:17.763978 7428 consensus_peers.cc:487] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 -> Peer 372fc71d57c940e3bacacd31cf83db4f (127.1.74.189:34295): Couldn't send request to peer 372fc71d57c940e3bacacd31cf83db4f. Status: Network error: Client connection negotiation failed: client connection to 127.1.74.189:34295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.188@KRBTEST.COM" created.
I20260619 22:21:17.765800 7905 master.cc:552] Master@127.1.74.189:34295 shutdown complete.
I20260619 22:21:17.765856 7905 master_runner.cc:305] Clearing existing system tablet
I20260619 22:21:17.766122 7905 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:17.766187 7905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:17.767635 7905 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 22:21:17.768616 8007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.768872 7905 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:17.769004 7905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:17.769040 7905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
uuid: "372fc71d57c940e3bacacd31cf83db4f"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "0768f672b457a076d2cfe0b27e2f4b81"
server_key_iv: "79b168251acff9c6277e26aa245dce90"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.771327 7905 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:17.773869 7905 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 22:21:17.773975 7905 log.cc:1196] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 22:21:17.774257 7905 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Deleting consensus metadata
I20260619 22:21:17.774446 7905 master_runner.cc:315] Copying system tablet from 127.1.74.190:39085
I20260619 22:21:17.775733 7905 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: tablet copy: Beginning tablet copy session from remote peer at address 127.1.74.190:39085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.188.keytab.
Entry for principal oryx/127.1.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.188.keytab.
I20260619 22:21:17.782100 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:34871
--webserver_interface=127.1.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.188.keytab
--principal=oryx/127.1.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:21:17.784749 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.776451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.1.74.190:39085)
0619 22:21:17.776606 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.776611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.776735 (+ 124us) server_negotiation.cc:407] Connection header received
0619 22:21:17.776924 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.776927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.776981 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.777085 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.777537 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.778109 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.779318 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.779741 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.779856 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.781104 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.781108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.781111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.781137 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.782950 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.783515 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.783520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.783523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.783612 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.783901 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.783906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.783907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.784095 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.784201 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.784460 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.784581 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:21:17.784759 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.776353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55214)
0619 22:21:17.776624 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.776639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.776748 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.777115 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.777117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.777276 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.777410 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.777421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.778221 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.778226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.779190 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.779204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.779860 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.779864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.779946 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.780568 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.780589 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.780956 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.783086 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.783092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.783097 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.783380 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.783718 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.783721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.783723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.783806 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.784190 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.784193 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.784295 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.784405 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.784559 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":112,"threads_started":1}
I20260619 22:21:17.785413 7461 tablet_copy_service.cc:140] P c3b1bc7f83df4b2ea1f56bc784bed180: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 372fc71d57c940e3bacacd31cf83db4f ({username='oryx', principal='oryx/127.1.74.189@KRBTEST.COM'} at 127.0.0.1:55214)
I20260619 22:21:17.785571 7461 tablet_copy_service.cc:161] P c3b1bc7f83df4b2ea1f56bc784bed180: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 372fc71d57c940e3bacacd31cf83db4f at {username='oryx', principal='oryx/127.1.74.189@KRBTEST.COM'} at 127.0.0.1:55214: session id = 372fc71d57c940e3bacacd31cf83db4f-00000000000000000000000000000000
I20260619 22:21:17.787969 7461 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 22:21:17.789855 7905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 22:21:17.792284 7905 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: tablet copy: Starting download of 0 data blocks...
I20260619 22:21:17.792479 7905 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: tablet copy: Starting download of 1 WAL segments...
I20260619 22:21:17.794374 7905 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 22:21:17.796630 7461 tablet_copy_service.cc:342] P c3b1bc7f83df4b2ea1f56bc784bed180: Request end of tablet copy session 372fc71d57c940e3bacacd31cf83db4f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.74.189@KRBTEST.COM'} at 127.0.0.1:55214
I20260619 22:21:17.796748 7461 tablet_copy_service.cc:433] P c3b1bc7f83df4b2ea1f56bc784bed180: ending tablet copy session 372fc71d57c940e3bacacd31cf83db4f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 372fc71d57c940e3bacacd31cf83db4f
I20260619 22:21:17.805184 7905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:17.806975 8015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.807050 8018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.807766 7905 server_base.cc:1034] running on GCE node
W20260619 22:21:17.811038 8016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.811335 7905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:17.811653 7905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:17.812804 7905 hybrid_clock.cc:648] HybridClock initialized: now 1781907677812785 us; error 43 us; skew 500 ppm
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:17.814424 7905 init.cc:377] Logged in from keytab as oryx/127.1.74.189@KRBTEST.COM (short username oryx)
I20260619 22:21:17.814935 7905 webserver.cc:460] Webserver started at http://127.1.74.189:34785/ using document root <none> and password file <none>
I20260619 22:21:17.815132 7905 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:17.815176 7905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:17.816471 7905 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 22:21:17.817500 8025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.817775 7905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 22:21:17.817848 7905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
uuid: "372fc71d57c940e3bacacd31cf83db4f"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "0768f672b457a076d2cfe0b27e2f4b81"
server_key_iv: "79b168251acff9c6277e26aa245dce90"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.817931 7905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:17.833498 7905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:17.835639 7905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:17.840380 7905 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.189:34295
I20260619 22:21:17.840418 8077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.189:34295 every 8 connection(s)
I20260619 22:21:17.840967 7905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
I20260619 22:21:17.842635 8078 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:21:17.843775 8078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Bootstrap starting.
I20260619 22:21:17.853853 8078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:21:17.854218 8078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: Bootstrap complete.
I20260619 22:21:17.854781 8078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } }
I20260619 22:21:17.854940 8078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 372fc71d57c940e3bacacd31cf83db4f, State: Initialized, Role: LEARNER
I20260619 22:21:17.855050 8078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } }
I20260619 22:21:17.855418 8081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } }
I20260619 22:21:17.855609 8081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: LEARNER
I20260619 22:21:17.855511 8078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:17.858831 8092 catalog_manager.cc:1259] Loaded cluster ID: 16beac010ee74b349c1a6c813c8158e9
I20260619 22:21:17.858922 8092 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: loading cluster ID for follower catalog manager: success
I20260619 22:21:17.861009 8092 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: acquiring CA information for follower catalog manager: success
I20260619 22:21:17.862423 8092 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 22:21:17.915465 8014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:17.915755 8014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:17.915824 8014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:17.920307 8014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:17.920372 8014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:17.920393 8014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:17.920411 8014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:17.920465 8014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:17.926061 8014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.188.keytab
--principal=oryx/127.1.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:34871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:17.927199 8014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:17.928275 8014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:17.936817 8100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.936800 8098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:17.936802 8097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:17.937152 8014 server_base.cc:1034] running on GCE node
I20260619 22:21:17.937664 8014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:17.938787 8014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:17.940027 8014 hybrid_clock.cc:648] HybridClock initialized: now 1781907677940007 us; error 30 us; skew 500 ppm
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:17.942895 8014 init.cc:377] Logged in from keytab as oryx/127.1.74.188@KRBTEST.COM (short username oryx)
I20260619 22:21:17.943825 8014 webserver.cc:460] Webserver started at http://127.1.74.188:43419/ using document root <none> and password file <none>
I20260619 22:21:17.944386 8014 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:17.944476 8014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:17.944712 8014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:17.946997 8014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/instance:
uuid: "9990ef2bf21245a49dbb64980e05a80a"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "00077559b87ec199aca24349919b2c2f"
server_key_iv: "ac0a422416ab3fed1d7d4e67a89fd60a"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.947649 8014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal/instance:
uuid: "9990ef2bf21245a49dbb64980e05a80a"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "00077559b87ec199aca24349919b2c2f"
server_key_iv: "ac0a422416ab3fed1d7d4e67a89fd60a"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.951431 8014 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:21:17.953894 8107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:17.955178 8014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:17.955312 8014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
uuid: "9990ef2bf21245a49dbb64980e05a80a"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "00077559b87ec199aca24349919b2c2f"
server_key_iv: "ac0a422416ab3fed1d7d4e67a89fd60a"
server_key_version: "encryptionkey@0"
I20260619 22:21:17.955423 8014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:17.966361 8014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:17.969846 8014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:17.970050 8014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:17.977422 8014 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.188:34871
I20260619 22:21:17.977695 8159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.188:34871 every 8 connection(s)
I20260619 22:21:17.978475 8014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
I20260619 22:21:17.979552 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 8014
I20260619 22:21:17.981829 8160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:17.986195 8160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.74.188@KRBTEST.COM
Jun 19 22:21:17 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.188@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:17.996536 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.981064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.1.74.188:34871)
0619 22:21:17.981494 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.981509 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.981544 (+ 35us) server_negotiation.cc:407] Connection header received
0619 22:21:17.982302 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.982320 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.982657 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.983019 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.983891 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.984907 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.986087 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.986700 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.986888 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.990097 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.990109 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.990118 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.990145 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.992851 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.993481 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.993489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.993498 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.993674 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.994021 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.994026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.994028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.994693 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.994978 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.995358 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.995690 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":136,"threads_started":1}
I20260619 22:21:17.998826 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.988470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.1.74.190:39085)
0619 22:21:17.988610 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:17.988615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:17.988944 (+ 329us) server_negotiation.cc:407] Connection header received
0619 22:21:17.989144 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:17.989147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:17.989205 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:17.989285 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:17.990050 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.990582 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.991609 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.991849 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.991965 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.994658 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:17.994662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:17.994664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:17.994687 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:17.996749 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.997257 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.997260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.997262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.997356 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:17.997813 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:17.997817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:17.997819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:17.998027 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:17.998147 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:17.998471 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 22:21:17.998598 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 22:21:17.998871 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:17.988285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55218)
0619 22:21:17.988807 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:17.988864 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:17.989028 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:17.989333 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:17.989339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:17.989797 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:17.989925 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.989930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.990711 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.990714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.991470 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:17.991480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:17.991967 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:17.991971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:17.992107 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:17.992779 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:17.992794 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:17.994487 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:17.996888 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.996898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.996907 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.997147 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.997597 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:17.997601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:17.997603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:17.997700 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:17.998147 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:17.998155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:17.998270 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 22:21:17.998414 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 22:21:17.998611 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":174,"threads_started":1}
I20260619 22:21:18.000226 8160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:21:18.002451 7632 heartbeater.cc:503] Master 127.1.74.190:39085 was elected leader, sending a full tablet report...
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.188@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
I20260619 22:21:18.009825 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.000783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42020)
0619 22:21:18.000898 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.000914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.000994 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.001415 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.001418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.001595 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.001694 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.001699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.002763 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.002769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.003453 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.003462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.004132 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.004135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.004220 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.004883 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.004895 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.006239 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.008320 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.008323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.008324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.008557 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.008855 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.008857 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.008858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.008941 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.009355 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.009358 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.009463 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.009561 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.009693 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 22:21:18.009862 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.000924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.1.74.189:34295)
0619 22:21:18.001213 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.001217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.001228 (+ 11us) server_negotiation.cc:407] Connection header received
0619 22:21:18.001262 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.001264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.001301 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.001401 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.001811 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.002634 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.003623 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.004005 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.004108 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.006398 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.006401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.006403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.006429 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.008180 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.008671 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.008674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.008679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.008762 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.009038 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.009040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.009041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.009227 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.009305 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.009628 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.009740 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
I20260619 22:21:18.010730 8160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907677, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.188@KRBTEST.COM for oryx/127.1.74.188@KRBTEST.COM
I20260619 22:21:18.020130 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.011331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59534)
0619 22:21:18.011553 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.011576 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.011649 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.011969 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.011971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.012122 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.012197 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.012202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.013338 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.013341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.013858 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.013866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.014357 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.014360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.014424 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.015012 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.015024 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.016516 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.018368 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.018371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.018373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.018601 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.018914 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.018917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.018918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.019016 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.019496 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.019499 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.019583 (+ 84us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.019677 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.019791 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260619 22:21:18.020169 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.011479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.1.74.188:34871)
0619 22:21:18.011609 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.011613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.011627 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:18.011778 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.011780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.011818 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.011886 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.012331 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.013218 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.013983 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.014210 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.014306 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.016664 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.016667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.016669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.016689 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.018253 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.018695 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.018697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.018699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.018767 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.019126 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.019129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.019130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.019373 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.019481 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.019753 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.019865 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:21:18.023801 8160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Bootstrap starting.
I20260619 22:21:18.026072 8160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:18.026811 8160 log.cc:824] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:18.028812 8160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: No bootstrap required, opened a new log
I20260619 22:21:18.032145 8160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } }
I20260619 22:21:18.032388 8160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:18.032469 8160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9990ef2bf21245a49dbb64980e05a80a, State: Initialized, Role: FOLLOWER
I20260619 22:21:18.032914 8160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } }
I20260619 22:21:18.033443 8166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } } }
I20260619 22:21:18.033581 8166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:18.034183 8160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: configured and running, proceeding with master startup.
W20260619 22:21:18.039000 8177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:21:18.039103 8177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:21:18.042377 8014 master_runner.cc:428] Detected that this master 9990ef2bf21245a49dbb64980e05a80a is joining an existing cluster
I20260619 22:21:18.042448 8014 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.74.188:34871
I20260619 22:21:18.050570 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7503
I20260619 22:21:18.055487 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.045478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.1.74.188:34871)
0619 22:21:18.045598 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.045602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.046058 (+ 456us) server_negotiation.cc:407] Connection header received
0619 22:21:18.046185 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.046188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.046226 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.046272 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.046903 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.047691 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049018 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.049345 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049406 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.051797 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.051801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.051804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.051829 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.053471 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.053989 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.053992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.053994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.054085 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.054409 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.054412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.054414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.054645 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.054742 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.055183 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.055308 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 22:21:18.055532 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.045288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59550)
0619 22:21:18.045976 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.045990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.046100 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.046393 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.046397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.046632 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.046782 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.046790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.047890 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.047893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.048906 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.048916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.049558 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.050353 (+ 792us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.051143 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.051163 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.051635 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.053597 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.053603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.053608 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.053881 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.054188 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.054194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.054199 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.054305 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.054754 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.054760 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.054904 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.055023 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.055211 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":730,"thread_start_us":73,"threads_started":1}
I20260619 22:21:18.059151 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:40109
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:39085
--webserver_port=46443
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:21:18.071733 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.045392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.1.74.190:39085)
0619 22:21:18.045521 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.045527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.045542 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:18.045589 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.045592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.045672 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.045757 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.046470 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.047349 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.048993 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.049312 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049405 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.067583 (+ 18178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.067588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.067591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.067624 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.069565 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.070311 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.070315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.070318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.070410 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.070715 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.070718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.070720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.070977 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.071060 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.071418 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.071546 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:21:18.072314 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.044854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55234)
0619 22:21:18.045317 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.045333 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.045427 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.045884 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.045915 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.046162 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.046322 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.046330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.047494 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.047498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.048852 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.048863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.049559 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.049678 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.050213 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.050230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.067329 (+ 17099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.069695 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.069699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.069701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.069943 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.070526 (+ 583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.070529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.070531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.070616 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.071132 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.071135 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.071243 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.071960 (+ 717us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.072129 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":7,"thread_start_us":98,"threads_started":1}
I20260619 22:21:18.072896 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.045021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42028)
0619 22:21:18.045772 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.045786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.045911 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.046277 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.046279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.046529 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.046717 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.046725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.047890 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.047893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.049141 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.049148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.049824 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.049926 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.067527 (+ 17601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.067551 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.068353 (+ 802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.071068 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.071072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.071074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.071304 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.071599 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.071602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.071605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.071682 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.072375 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.072379 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.072475 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.072576 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.072709 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":39,"thread_start_us":112,"threads_started":1}
I20260619 22:21:18.074237 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.044908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.1.74.189:34295)
0619 22:21:18.045067 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.045072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.045849 (+ 777us) server_negotiation.cc:407] Connection header received
0619 22:21:18.046026 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.046031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.046086 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.046140 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.046873 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.047699 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049249 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.049613 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.049685 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.068577 (+ 18892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.068581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.068585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.068617 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.070930 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.071415 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.071418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.071420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.071502 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.071775 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.071778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.071779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.072001 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.072088 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.073908 (+ 1820us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.074060 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907676, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.190@KRBTEST.COM for oryx/127.1.74.188@KRBTEST.COM
I20260619 22:21:18.086093 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.076544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59566)
0619 22:21:18.076657 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.076671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.076748 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.077040 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.077044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.077230 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.077341 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.077349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.078359 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.078363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.079438 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.079451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.080033 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.080037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.080136 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.080758 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.080795 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.082466 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.084458 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.084461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.084464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.084722 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.085021 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.085024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.085026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.085117 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.085523 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.085526 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.085631 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.085738 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.085896 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 22:21:18.086513 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.076655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.1.74.188:34871)
0619 22:21:18.076756 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.076760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.076774 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:18.076847 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.076850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.076898 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.076985 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.077466 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.078246 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.079569 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.079910 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.080025 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.082631 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.082634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.082637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.082664 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.084329 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.084825 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.084829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.084831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.084914 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.085210 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.085213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.085214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.085423 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.085518 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.085801 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.086021 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 22:21:18.087790 7441 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.74.188:34871: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a"
member_type: NON_VOTER
last_known_addr {
host: "127.1.74.188"
port: 34871
}
attrs {
promote: true
}
}
dest_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180"
cas_config_opid_index: 5
I20260619 22:21:18.088405 7441 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } }
I20260619 22:21:18.089658 8002 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.091310 8002 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.74.188:34871
I20260619 22:21:18.091498 8002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.091589 8002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.092257 8134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:18.093668 8134 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Refusing update from remote peer c3b1bc7f83df4b2ea1f56bc784bed180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 22:21:18.094410 8014 master.cc:537] Master@127.1.74.188:34871 shutting down...
I20260619 22:21:18.094417 8001 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 22:21:18.096845 7427 consensus_peers.cc:487] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 -> Peer 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188:34871): Couldn't send request to peer 9990ef2bf21245a49dbb64980e05a80a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 22:21:18.103168 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.091156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42032)
0619 22:21:18.097148 (+ 5992us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.097163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.097254 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.097667 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.097670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.097686 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.097778 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.097784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.099000 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.099003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.100848 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.100862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.102648 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.102652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.102749 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.102802 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.102905 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.103003 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5921}
I20260619 22:21:18.103614 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.091407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.1.74.189:34295)
0619 22:21:18.097052 (+ 5645us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.097057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.097336 (+ 279us) server_negotiation.cc:407] Connection header received
0619 22:21:18.097391 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.097394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.097454 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.097537 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.097965 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.098774 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.101003 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.102313 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.102421 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.102580 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.103412 (+ 832us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.103497 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5528}
I20260619 22:21:18.103834 8014 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:18.104096 8052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 LEARNER]: Refusing update from remote peer c3b1bc7f83df4b2ea1f56bc784bed180: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 22:21:18.104241 8014 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:18.104382 8014 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: stopping tablet replica
I20260619 22:21:18.104559 8001 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 22:21:18.106448 8052 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.109359 8081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b1bc7f83df4b2ea1f56bc784bed180. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.109499 8081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: LEARNER
I20260619 22:21:18.109593 8081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: NON_VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: true } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.109669 8081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: LEARNER
I20260619 22:21:18.110652 8014 master.cc:552] Master@127.1.74.188:34871 shutdown complete.
I20260619 22:21:18.110713 8014 master_runner.cc:305] Clearing existing system tablet
I20260619 22:21:18.111019 8014 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:18.111092 8014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:18.112237 8001 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: attempting to promote NON_VOTER 372fc71d57c940e3bacacd31cf83db4f to VOTER
I20260619 22:21:18.112385 8014 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:18.112720 8001 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } }
I20260619 22:21:18.114313 8052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 LEARNER]: Refusing update from remote peer c3b1bc7f83df4b2ea1f56bc784bed180: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 22:21:18.114693 8002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 22:21:18.115023 8190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.115218 8014 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:18.115322 8014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:18.115353 8014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
uuid: "9990ef2bf21245a49dbb64980e05a80a"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "00077559b87ec199aca24349919b2c2f"
server_key_iv: "ac0a422416ab3fed1d7d4e67a89fd60a"
server_key_version: "encryptionkey@0"
I20260619 22:21:18.116365 8002 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 372fc71d57c940e3bacacd31cf83db4f (127.1.74.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.117496 8014 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:18.117877 8002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.117976 8002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.118124 8002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.118198 8002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.119104 8052 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 372fc71d57c940e3bacacd31cf83db4f (127.1.74.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.120689 8014 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 22:21:18.120771 8014 log.cc:1196] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 22:21:18.121003 8014 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Deleting consensus metadata
I20260619 22:21:18.121229 8014 master_runner.cc:315] Copying system tablet from 127.1.74.190:39085
I20260619 22:21:18.121331 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.113812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59570)
0619 22:21:18.116098 (+ 2286us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.121240 (+ 5142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:34871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2185}
I20260619 22:21:18.122419 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.121808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59572)
0619 22:21:18.122282 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.122336 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:34871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":101,"threads_started":1}
I20260619 22:21:18.122381 8014 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: tablet copy: Beginning tablet copy session from remote peer at address 127.1.74.190:39085
W20260619 22:21:18.122503 7427 consensus_peers.cc:487] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 -> Peer 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188:34871): Couldn't send request to peer 9990ef2bf21245a49dbb64980e05a80a. Status: Network error: Client connection negotiation failed: client connection to 127.1.74.188:34871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 22:21:18.122825 8188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.123293 8188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:18.135687 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.123109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.1.74.190:39085)
0619 22:21:18.123231 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.123256 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.127178 (+ 3922us) server_negotiation.cc:407] Connection header received
0619 22:21:18.127323 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.127327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.127385 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.127480 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.128064 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.128975 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.130268 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.130613 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.130721 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.132152 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.132156 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.132159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.132187 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.133846 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.134361 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.134365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.134367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.134451 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.134725 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.134728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.134730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.134994 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.135098 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.135400 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.135527 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:18.135984 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.123119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55246)
0619 22:21:18.127055 (+ 3936us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.127083 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.127218 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.127522 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.127526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.127788 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.127919 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.127927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.129116 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.129122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.130155 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.130166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.130760 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.130767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.130904 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.131530 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.131545 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.131997 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.133962 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.133966 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.133968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.134268 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.134541 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.134543 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.134545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.134635 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.135093 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.135098 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.135205 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.135312 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.135464 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3848,"thread_start_us":64,"threads_started":1}
I20260619 22:21:18.136521 7461 tablet_copy_service.cc:140] P c3b1bc7f83df4b2ea1f56bc784bed180: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9990ef2bf21245a49dbb64980e05a80a ({username='oryx', principal='oryx/127.1.74.188@KRBTEST.COM'} at 127.0.0.1:55246)
I20260619 22:21:18.136619 7461 tablet_copy_service.cc:161] P c3b1bc7f83df4b2ea1f56bc784bed180: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9990ef2bf21245a49dbb64980e05a80a at {username='oryx', principal='oryx/127.1.74.188@KRBTEST.COM'} at 127.0.0.1:55246: session id = 9990ef2bf21245a49dbb64980e05a80a-00000000000000000000000000000000
I20260619 22:21:18.137758 7461 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 22:21:18.139228 8014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 22:21:18.141580 8014 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: tablet copy: Starting download of 0 data blocks...
I20260619 22:21:18.141754 8014 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: tablet copy: Starting download of 1 WAL segments...
I20260619 22:21:18.143453 8014 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 22:21:18.145337 7461 tablet_copy_service.cc:342] P c3b1bc7f83df4b2ea1f56bc784bed180: Request end of tablet copy session 9990ef2bf21245a49dbb64980e05a80a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.74.188@KRBTEST.COM'} at 127.0.0.1:55246
I20260619 22:21:18.145423 7461 tablet_copy_service.cc:433] P c3b1bc7f83df4b2ea1f56bc784bed180: ending tablet copy session 9990ef2bf21245a49dbb64980e05a80a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9990ef2bf21245a49dbb64980e05a80a
I20260619 22:21:18.152683 8014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:18.154739 8199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.154783 8200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:18.155314 8014 server_base.cc:1034] running on GCE node
W20260619 22:21:18.159029 8202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:18.159338 8014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:18.159685 8014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:18.160851 8014 hybrid_clock.cc:648] HybridClock initialized: now 1781907678160837 us; error 44 us; skew 500 ppm
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:18.162570 8014 init.cc:377] Logged in from keytab as oryx/127.1.74.188@KRBTEST.COM (short username oryx)
I20260619 22:21:18.163160 8014 webserver.cc:460] Webserver started at http://127.1.74.188:40081/ using document root <none> and password file <none>
I20260619 22:21:18.163358 8014 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:18.163406 8014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:18.164893 8014 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:18.166035 8209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.166340 8014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:18.166424 8014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
uuid: "9990ef2bf21245a49dbb64980e05a80a"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "00077559b87ec199aca24349919b2c2f"
server_key_iv: "ac0a422416ab3fed1d7d4e67a89fd60a"
server_key_version: "encryptionkey@0"
I20260619 22:21:18.166507 8014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:18.178179 8014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:18.180702 8014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:18.185966 8014 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.188:34871
I20260619 22:21:18.186681 8014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
I20260619 22:21:18.187022 8261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.188:34871 every 8 connection(s)
I20260619 22:21:18.188962 8262 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:21:18.190217 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Bootstrap starting.
I20260619 22:21:18.201988 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:21:18.202335 8262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Bootstrap complete.
I20260619 22:21:18.203044 8262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } }
I20260619 22:21:18.203199 8262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9990ef2bf21245a49dbb64980e05a80a, State: Initialized, Role: LEARNER
I20260619 22:21:18.203301 8262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } }
I20260619 22:21:18.203626 8268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.203750 8262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:18.203840 8268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: This master's current role is: LEARNER
I20260619 22:21:18.207408 8279 catalog_manager.cc:1259] Loaded cluster ID: 16beac010ee74b349c1a6c813c8158e9
I20260619 22:21:18.207548 8279 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: loading cluster ID for follower catalog manager: success
I20260619 22:21:18.209692 8279 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: acquiring CA information for follower catalog manager: success
W20260619 22:21:18.209595 8186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:18.209906 8186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:18.209985 8186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 22:21:18.211293 8279 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 22:21:18.216452 8186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:18.216563 8186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:18.216811 8186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:18.225739 8186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:40109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=46443
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:18.227237 8186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:18.228734 8186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:18.231205 7766 heartbeater.cc:503] Master 127.1.74.190:39085 was elected leader, sending a full tablet report...
W20260619 22:21:18.241644 8281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.241648 8284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.241845 8282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:18.242419 8186 server_base.cc:1034] running on GCE node
I20260619 22:21:18.242916 8186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:18.243732 8186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:18.244933 8186 hybrid_clock.cc:648] HybridClock initialized: now 1781907678244921 us; error 26 us; skew 500 ppm
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:18.247876 8186 init.cc:377] Logged in from keytab as oryx/127.1.74.129@KRBTEST.COM (short username oryx)
I20260619 22:21:18.248714 8186 webserver.cc:460] Webserver started at http://127.1.74.129:46443/ using document root <none> and password file <none>
I20260619 22:21:18.249241 8186 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:18.249312 8186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:18.253305 8186 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:21:18.255537 8291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.256887 8186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:18.256986 8186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "0362f0c1fc754d54be8d60b73956df39"
format_stamp: "Formatted at 2026-06-19 22:21:16 on dist-test-slave-k84h"
server_key: "b0329130b9646193e356c8c163ef7184"
server_key_iv: "4fd14868020026927703e5eddc6a57c9"
server_key_version: "encryptionkey@0"
I20260619 22:21:18.257593 8186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:18.271392 8186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:18.274169 8186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:18.274323 8186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:18.274871 8186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:18.275664 8186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:18.275708 8186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.275746 8186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:18.275766 8186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.285180 8186 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:40109
I20260619 22:21:18.285211 8404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:40109 every 8 connection(s)
I20260619 22:21:18.286144 8186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:18.292222 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 8186
I20260619 22:21:18.292412 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7637
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.188@KRBTEST.COM
I20260619 22:21:18.301638 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:36505
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:39085
--webserver_port=37889
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:21:18.305716 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.288016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:45163 (local address 127.1.74.189:34295)
0619 22:21:18.288142 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.288147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.289254 (+ 1107us) server_negotiation.cc:407] Connection header received
0619 22:21:18.290576 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.290582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.290645 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.290733 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.292098 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.292920 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295114 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.295545 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295669 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.300917 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.300923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.300929 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.300963 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.302871 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.303698 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.303703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.303705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.303799 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.304218 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.304223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.304226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.304475 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.304574 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.305400 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.305534 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 22:21:18.305744 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.288171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:49153 (local address 127.1.74.190:39085)
0619 22:21:18.288279 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.288283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.289563 (+ 1280us) server_negotiation.cc:407] Connection header received
0619 22:21:18.290632 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.290635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.290690 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.290763 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.292104 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.292906 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295078 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.295415 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295494 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.300591 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.300595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.300597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.300626 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.302873 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.303720 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.303724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.303726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.303828 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.304241 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.304247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.304252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.304511 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.304613 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.305417 (+ 804us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.305558 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 22:21:18.307058 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.289796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:47363 (local address 127.1.74.188:34871)
0619 22:21:18.290696 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.290720 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.290735 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:18.290776 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.290779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.290826 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.290938 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.292095 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.293155 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295068 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.295416 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295503 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.301602 (+ 6099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.301606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.301609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.301642 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.304146 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.304661 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.304664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.304666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.304795 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.305205 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.305208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.305211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.305689 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.306438 (+ 749us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.306574 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.306769 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":86,"threads_started":1}
I20260619 22:21:18.307466 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.288469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.1.74.129:45163)
0619 22:21:18.289098 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.289154 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.290293 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.290963 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.290971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.291434 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.291947 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.291957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.293085 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.293091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.294975 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.294988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.295821 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.296010 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.296869 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.296912 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.300756 (+ 3844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.303008 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.303014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.303035 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.303562 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.303913 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.303917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.303921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.304109 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.304654 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.304664 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.305103 (+ 439us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.305878 (+ 775us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.306588 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":541,"spinlock_wait_cycles":16000,"thread_start_us":155,"threads_started":1}
I20260619 22:21:18.307474 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.288469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:49153)
0619 22:21:18.289481 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.289494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.290293 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.290964 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.290972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.291446 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.291928 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.291947 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.293082 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.293085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.294931 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.294943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.295669 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.295999 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.296909 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.296935 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.300261 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.303008 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.303014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.303035 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.303574 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.303981 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.303985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.303987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.304115 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.304671 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.304674 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.305102 (+ 428us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.305981 (+ 879us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.306282 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":688,"spinlock_wait_cycles":124160,"thread_start_us":736,"threads_started":1}
I20260619 22:21:18.308164 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.289701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.1.74.129:47363)
0619 22:21:18.289977 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.289997 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.290333 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.290964 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.290972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.291449 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.291932 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.291947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.293269 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.293272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.294951 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.294957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.295669 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.295673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.295999 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.297028 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.297044 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.301352 (+ 4308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.304273 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.304276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.304278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.304520 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.304959 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.304962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.304964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.305072 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.305901 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.305905 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.306077 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.306238 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.306432 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":610,"spinlock_wait_cycles":19840,"thread_start_us":106,"threads_started":1}
I20260619 22:21:18.308820 8407 heartbeater.cc:347] Connected to a master server at 127.1.74.189:34295
I20260619 22:21:18.309068 8407 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.309185 8405 heartbeater.cc:347] Connected to a master server at 127.1.74.188:34871
I20260619 22:21:18.309280 8405 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.309321 8406 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39085
I20260619 22:21:18.309378 8406 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.309644 8407 heartbeater.cc:511] Master 127.1.74.189:34295 requested a full tablet report, sending...
I20260619 22:21:18.309710 8406 heartbeater.cc:511] Master 127.1.74.190:39085 requested a full tablet report, sending...
I20260619 22:21:18.309643 8405 heartbeater.cc:511] Master 127.1.74.188:34871 requested a full tablet report, sending...
I20260619 22:21:18.311100 7441 ts_manager.cc:194] Re-registered known tserver with Master: 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109)
I20260619 22:21:18.311364 8042 ts_manager.cc:194] Registered new tserver with Master: 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109)
I20260619 22:21:18.311546 8225 ts_manager.cc:194] Registered new tserver with Master: 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109)
I20260619 22:21:18.311918 7441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:49153
W20260619 22:21:18.425001 8413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:18.425315 8413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:18.425390 8413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:18.429567 8413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:18.429656 8413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:18.429744 8413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:18.435231 8413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:36505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=37889
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:18.436442 8413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:18.437505 8413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:18.444949 7900 heartbeater.cc:503] Master 127.1.74.190:39085 was elected leader, sending a full tablet report...
W20260619 22:21:18.447274 8419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.447392 8418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.447291 8421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:18.447875 8413 server_base.cc:1034] running on GCE node
I20260619 22:21:18.448261 8413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:18.448909 8413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:18.450151 8413 hybrid_clock.cc:648] HybridClock initialized: now 1781907678450116 us; error 51 us; skew 500 ppm
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:18.453263 8413 init.cc:377] Logged in from keytab as oryx/127.1.74.130@KRBTEST.COM (short username oryx)
I20260619 22:21:18.454178 8413 webserver.cc:460] Webserver started at http://127.1.74.130:37889/ using document root <none> and password file <none>
I20260619 22:21:18.454741 8413 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:18.454862 8413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:18.458925 8413 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 22:21:18.461074 8428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.462352 8413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 22:21:18.462489 8413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "247dc117e29441ee9cb12818c64ede15"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "e305eeb6d206bb1fda2410432ca2d66a"
server_key_iv: "d44f90f8653f69b3275273826aa368de"
server_key_version: "encryptionkey@0"
I20260619 22:21:18.463114 8413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:18.476202 8413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:18.479115 8413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:18.479324 8413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:18.479905 8413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:18.480744 8413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:18.480815 8413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.480892 8413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:18.480934 8413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.489799 8413 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:36505
I20260619 22:21:18.489848 8541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:36505 every 8 connection(s)
I20260619 22:21:18.490836 8413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:18.493470 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 8413
I20260619 22:21:18.493700 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7771
I20260619 22:21:18.502243 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:33087
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:39085
--webserver_port=38987
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.188@KRBTEST.COM
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:18.509608 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.494381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:42469 (local address 127.1.74.188:34871)
0619 22:21:18.494545 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.494553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.494662 (+ 109us) server_negotiation.cc:407] Connection header received
0619 22:21:18.495369 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.495373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.495421 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.495472 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.497102 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.497931 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500123 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.500483 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500559 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.504922 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.504929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.504933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.504962 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.507258 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.508101 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.508104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.508106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.508185 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.508463 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.508466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.508468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.508686 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.508759 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.509288 (+ 529us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.509430 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:21:18.509737 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.492653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:33561 (local address 127.1.74.189:34295)
0619 22:21:18.492775 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.492779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.494155 (+ 1376us) server_negotiation.cc:407] Connection header received
0619 22:21:18.495396 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.495399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.495455 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.495533 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.497107 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.498016 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500136 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.500593 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500707 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.505164 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.505171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.505177 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.505210 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.507009 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.507757 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.507761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.507763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.507848 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.508264 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.508267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.508270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.508525 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.508613 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.509480 (+ 867us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.509588 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:21:18.510828 8546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.493057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:48731)
0619 22:21:18.493579 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.493628 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.494744 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.495439 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.495449 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.496032 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.496934 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.496953 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.498110 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.498113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.499980 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.499992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.500765 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.502281 (+ 1512us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.502768 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.502782 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.504940 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.507461 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.507465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.507470 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.507824 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.508166 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.508172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.508176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.508271 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.508816 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.508822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.509015 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.509217 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.509617 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":462,"spinlock_wait_cycles":5632,"thread_start_us":716,"threads_started":1}
I20260619 22:21:18.511336 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.492633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:48731 (local address 127.1.74.190:39085)
0619 22:21:18.492765 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.492770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.493729 (+ 959us) server_negotiation.cc:407] Connection header received
0619 22:21:18.495084 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.495090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.495153 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.495246 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.497102 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.497931 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500124 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.500483 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500559 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.505092 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.505098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.505105 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.505139 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.507320 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.507952 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.507958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.507961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.508052 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.508384 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.508390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.508394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.508661 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.508767 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.510973 (+ 2206us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.511113 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:21:18.510816 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.493056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.1.74.130:33561)
0619 22:21:18.494068 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.494084 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.494744 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.495584 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.495589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.496032 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.496946 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.496961 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.498155 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.498162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.499988 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.500002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.500863 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.501011 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.501883 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.501904 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.504977 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.507162 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.507169 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.507189 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.507659 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.507951 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.507954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.507955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.508155 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.508665 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.508672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.509011 (+ 339us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.509216 (+ 205us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.510015 (+ 799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":984,"spinlock_wait_cycles":43392,"thread_start_us":165,"threads_started":1}
I20260619 22:21:18.511821 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.494151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.1.74.130:42469)
0619 22:21:18.494574 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.494586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.495245 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.496163 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.496166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.496317 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.496933 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.496953 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.498110 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.498113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.499992 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.499999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.500765 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.500769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.501018 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.501993 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.502007 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.504699 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.507373 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.507376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.507378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.507994 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.508289 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.508292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.508294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.508366 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.508807 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.508810 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.509017 (+ 207us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.510191 (+ 1174us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.510315 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":585,"spinlock_wait_cycles":48512,"thread_start_us":81,"threads_started":1}
I20260619 22:21:18.512677 8544 heartbeater.cc:347] Connected to a master server at 127.1.74.189:34295
I20260619 22:21:18.512914 8544 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.513176 8542 heartbeater.cc:347] Connected to a master server at 127.1.74.188:34871
I20260619 22:21:18.513249 8542 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.513367 8543 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39085
I20260619 22:21:18.513422 8543 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.513531 8542 heartbeater.cc:511] Master 127.1.74.188:34871 requested a full tablet report, sending...
I20260619 22:21:18.513535 8543 heartbeater.cc:511] Master 127.1.74.190:39085 requested a full tablet report, sending...
I20260619 22:21:18.513536 8544 heartbeater.cc:511] Master 127.1.74.189:34295 requested a full tablet report, sending...
I20260619 22:21:18.514878 8042 ts_manager.cc:194] Registered new tserver with Master: 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505)
I20260619 22:21:18.514884 8225 ts_manager.cc:194] Registered new tserver with Master: 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505)
I20260619 22:21:18.515250 7441 ts_manager.cc:194] Re-registered known tserver with Master: 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505)
I20260619 22:21:18.515831 7441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:48731
W20260619 22:21:18.619052 8549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:18.619314 8549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:18.619357 8549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:18.623811 8549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:18.623878 8549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:18.623939 8549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:18.629462 8549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:33087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=38987
--tserver_master_addrs=127.1.74.190:39085,127.1.74.189:34295,127.1.74.188:34871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:18.630456 8549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:18.631489 8549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:18.640779 8554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.640735 8555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:18.640733 8557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:18.641094 8549 server_base.cc:1034] running on GCE node
I20260619 22:21:18.641588 8549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:18.642221 8549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:18.643426 8549 hybrid_clock.cc:648] HybridClock initialized: now 1781907678643397 us; error 56 us; skew 500 ppm
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:18.646287 8549 init.cc:377] Logged in from keytab as oryx/127.1.74.131@KRBTEST.COM (short username oryx)
I20260619 22:21:18.647360 8549 webserver.cc:460] Webserver started at http://127.1.74.131:38987/ using document root <none> and password file <none>
I20260619 22:21:18.647965 8549 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:18.648065 8549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:18.652207 8549 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:21:18.654268 8564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.656032 8549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:18.656183 8549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "e6fc676f2fb7487c93e6f8f85d30de64"
format_stamp: "Formatted at 2026-06-19 22:21:17 on dist-test-slave-k84h"
server_key: "a16a20b2ed03220c5d94e3eec898803f"
server_key_iv: "e0852d44bd8fc48ec740f43fe71d621d"
server_key_version: "encryptionkey@0"
I20260619 22:21:18.656766 8549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:18.677568 8549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:18.680722 8549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:18.680959 8549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:18.681584 8549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:18.682616 8549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:18.682691 8549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.682770 8549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:18.682824 8549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:18.693046 8549 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:33087
I20260619 22:21:18.693069 8677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:33087 every 8 connection(s)
I20260619 22:21:18.694063 8549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:18.699877 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 8549
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.189@KRBTEST.COM
Jun 19 22:21:18 dist-test-slave-k84h krb5kdc[7396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907678, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.188@KRBTEST.COM
I20260619 22:21:18.712309 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.695925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:41231 (local address 127.1.74.189:34295)
0619 22:21:18.696101 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.696107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.697178 (+ 1071us) server_negotiation.cc:407] Connection header received
0619 22:21:18.698670 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.698673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.698720 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.698804 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.700869 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.701634 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.702721 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.703055 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.703467 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.707785 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.707789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.707791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.707819 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.710132 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710591 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.710594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.710596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.710682 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710999 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.711002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.711004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.711212 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.711298 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.712020 (+ 722us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.712135 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 22:21:18.713228 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.695934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:44459 (local address 127.1.74.190:39085)
0619 22:21:18.696201 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.696206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.697422 (+ 1216us) server_negotiation.cc:407] Connection header received
0619 22:21:18.698292 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.698296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.698355 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.698434 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.699898 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.700506 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.702035 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.702284 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.702417 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.706227 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.706231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.706234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.706266 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.708718 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710259 (+ 1541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.710264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.710266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.710346 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.711122 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.711128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.711133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.711380 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.711485 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.712964 (+ 1479us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.713083 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260619 22:21:18.713481 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.697346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.1.74.131:48589)
0619 22:21:18.697635 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.697647 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.698077 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.698646 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.698654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.699220 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.699749 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.699760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.700837 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.700843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.701900 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.701909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.704103 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.704107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.704223 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.704665 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.704681 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.706517 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.709219 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.709222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.709224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.709664 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.709939 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.709955 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.709957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.710151 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.712008 (+ 1857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.712011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.712110 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.712221 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.712379 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":67,"spinlock_wait_cycles":40448,"thread_start_us":84,"threads_started":1}
I20260619 22:21:18.713487 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.696358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.1.74.131:41231)
0619 22:21:18.696958 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.697035 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.698051 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.698867 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.698871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.699217 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.700741 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.700747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.701769 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.701772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.702586 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.702596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.703654 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.703658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.703752 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.704239 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.704254 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.707606 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.710262 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.710267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.710486 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.710783 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.710787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.710892 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.711352 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.711359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.711718 (+ 359us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.712512 (+ 794us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.712645 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":1135,"spinlock_wait_cycles":29184,"thread_start_us":663,"threads_started":1}
I20260619 22:21:18.714365 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.697454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:48589 (local address 127.1.74.188:34871)
0619 22:21:18.697582 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.697587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.697728 (+ 141us) server_negotiation.cc:407] Connection header received
0619 22:21:18.698389 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.698392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.698462 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.698548 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.699911 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.700690 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.703422 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.703797 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.703903 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.706679 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.706686 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.706691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.706720 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.708768 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.709760 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.709764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.709766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.709847 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710447 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.710450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.710452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.711861 (+ 1409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.711961 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.714075 (+ 2114us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.714200 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:21:18.714648 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.696358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:44459)
0619 22:21:18.697338 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.697351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.698051 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.698648 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.698655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.699217 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:18.699744 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.699758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.700645 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.700648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.701913 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.701921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.702609 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.702614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.702819 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.703511 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:18.703530 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:18.705980 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:18.708845 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.708852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.708870 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.709511 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.710890 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:18.710894 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:18.710896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:18.711003 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:18.711638 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:18.711641 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:18.711718 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.711935 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.712300 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":47,"spinlock_wait_cycles":26880,"thread_start_us":142,"threads_started":1}
I20260619 22:21:18.714896 8680 heartbeater.cc:347] Connected to a master server at 127.1.74.189:34295
I20260619 22:21:18.715131 8680 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.715766 8680 heartbeater.cc:511] Master 127.1.74.189:34295 requested a full tablet report, sending...
I20260619 22:21:18.716380 8678 heartbeater.cc:347] Connected to a master server at 127.1.74.188:34871
I20260619 22:21:18.716464 8678 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.716655 8678 heartbeater.cc:511] Master 127.1.74.188:34871 requested a full tablet report, sending...
I20260619 22:21:18.717043 8042 ts_manager.cc:194] Registered new tserver with Master: e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087)
I20260619 22:21:18.718173 8225 ts_manager.cc:194] Registered new tserver with Master: e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087)
I20260619 22:21:18.718369 8679 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39085
I20260619 22:21:18.718461 8679 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:18.718627 8679 heartbeater.cc:511] Master 127.1.74.190:39085 requested a full tablet report, sending...
I20260619 22:21:18.718672 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.703226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55256 (local address 127.1.74.190:39085)
0619 22:21:18.705186 (+ 1960us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.705198 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.705211 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:21:18.705247 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.705250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.705303 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.705404 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.706195 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.707058 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.709190 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.709581 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.710659 (+ 1078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.711927 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.711930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.711933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.711955 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.716056 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.716662 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.716665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.716667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.716802 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.717174 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.717178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.717181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.717416 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.717598 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.718279 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.718468 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1879,"thread_start_us":70,"threads_started":1}
I20260619 22:21:18.718998 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.703256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.1.74.189:34295)
0619 22:21:18.704579 (+ 1323us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.704585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.704601 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:21:18.704640 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.704643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.704729 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.704821 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.706161 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.706989 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.709686 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.710051 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.710145 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.714320 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.714324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.714326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.714349 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.716902 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.717372 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.717376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.717378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.717505 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.717825 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.717828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.717830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.718087 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.718208 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.718648 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.718771 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1223,"thread_start_us":93,"threads_started":1}
I20260619 22:21:18.719724 7441 ts_manager.cc:194] Re-registered known tserver with Master: e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087)
I20260619 22:21:18.720494 7441 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:44459
I20260619 22:21:18.724121 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.702810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.1.74.188:34871)
0619 22:21:18.703060 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.703073 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.707573 (+ 4500us) server_negotiation.cc:407] Connection header received
0619 22:21:18.707611 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.707614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.707666 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.707752 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:18.708722 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.709545 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.710761 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.711121 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.711212 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.720423 (+ 9211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:18.720429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:18.720436 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:18.720468 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:18.722328 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.722800 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.722803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.722805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.722934 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:18.723237 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:18.723240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:18.723242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:18.723422 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:18.723570 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.723811 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.723939 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260619 22:21:18.722231 7441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:18.725898 7441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:18.729074 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.717120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59590)
0619 22:21:18.720861 (+ 3741us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.720877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.721002 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.721510 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.721512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.721527 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.721593 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.721597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.727006 (+ 5409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.727010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.727804 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.727815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.728689 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.728692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.728796 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.728827 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.728897 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.728961 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3669,"thread_start_us":72,"threads_started":1}
I20260619 22:21:18.729107 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.717553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.1.74.188:34871)
0619 22:21:18.717967 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.717971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.720937 (+ 2966us) server_negotiation.cc:407] Connection header received
0619 22:21:18.721246 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.721249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.721314 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.721386 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.721688 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.722581 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.727965 (+ 5384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.728564 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.728662 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.728821 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.728923 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.728994 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336}
I20260619 22:21:18.729655 8236 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 22:21:18.730156 8697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b1bc7f83df4b2ea1f56bc784bed180. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.730425 8697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: This master's current role is: LEARNER
I20260619 22:21:18.730967 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: NON_VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: true } } }
I20260619 22:21:18.731102 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.734377 8700 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: attempting to promote NON_VOTER 9990ef2bf21245a49dbb64980e05a80a to VOTER
I20260619 22:21:18.734762 8700 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } }
I20260619 22:21:18.735925 8236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 LEARNER]: Refusing update from remote peer c3b1bc7f83df4b2ea1f56bc784bed180: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 22:21:18.735944 8052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Refusing update from remote peer c3b1bc7f83df4b2ea1f56bc784bed180: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 22:21:18.736272 8191 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 22:21:18.736451 8699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 22:21:18.738742 8191 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.739038 8052 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.739221 8236 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.740335 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.740435 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.740564 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.740639 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.740880 8191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.740978 8191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:18.741127 8697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.741241 8697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:18.741163 8707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:18.741292 8707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:18.747874 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.742594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36505 (local address 127.0.0.1:56998)
0619 22:21:18.742711 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.742727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.742807 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.743776 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.743779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.743796 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.743897 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.743903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.745124 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.745127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.746360 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.746370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.747464 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.747468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.747548 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.747569 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.747632 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.747720 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":3}
I20260619 22:21:18.747913 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.742738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.1.74.130:36505)
0619 22:21:18.743257 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.743272 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.743297 (+ 25us) server_negotiation.cc:407] Connection header received
0619 22:21:18.743381 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.743386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.743623 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.743814 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.743992 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.745004 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.746521 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.747339 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.747456 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.747615 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.747668 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.747781 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":164,"threads_started":1}
I20260619 22:21:18.749271 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.743209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:33087 (local address 127.0.0.1:37574)
0619 22:21:18.744246 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.744257 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.744490 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.745134 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.745137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.745154 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.745237 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.745242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.746521 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.746524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.747650 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.747660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.748834 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.748838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.748927 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.748996 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.749069 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.749138 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":24,"thread_start_us":89,"threads_started":1}
I20260619 22:21:18.749382 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.743374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.1.74.131:33087)
0619 22:21:18.743684 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.743695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.744369 (+ 674us) server_negotiation.cc:407] Connection header received
0619 22:21:18.744763 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.744769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.744926 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.745060 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.745350 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.746401 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.747786 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.748549 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.748661 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.748804 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.749118 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.749220 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":96,"threads_started":1}
I20260619 22:21:18.749571 8712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.743463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.1.74.129:40109)
0619 22:21:18.743841 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.743848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.744056 (+ 208us) server_negotiation.cc:407] Connection header received
0619 22:21:18.744189 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.744194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.744392 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.744551 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.744937 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.745810 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.747001 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.748664 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.748753 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.748859 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.749336 (+ 477us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.749436 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":90,"threads_started":1}
I20260619 22:21:18.749797 8709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.743071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40109 (local address 127.0.0.1:38586)
0619 22:21:18.743947 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.743958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.744079 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.744705 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.744708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.744725 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.744812 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.744817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.745922 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.745927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.746835 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.746844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.748923 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.748925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.749040 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.749092 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.749162 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.749228 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":25,"thread_start_us":102,"threads_started":1}
I20260619 22:21:18.751737 8476 tablet_service.cc:1467] Processing CreateTablet for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea (DEFAULT_TABLE table=test-table [id=bdeeb0b104fc4e8fbcd907bbcdca0d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:18.752899 8476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:18.752595 8612 tablet_service.cc:1467] Processing CreateTablet for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea (DEFAULT_TABLE table=test-table [id=bdeeb0b104fc4e8fbcd907bbcdca0d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:18.752774 8339 tablet_service.cc:1467] Processing CreateTablet for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea (DEFAULT_TABLE table=test-table [id=bdeeb0b104fc4e8fbcd907bbcdca0d74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:18.753513 8339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:18.753567 8612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:18.760390 8713 tablet_bootstrap.cc:492] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Bootstrap starting.
I20260619 22:21:18.762013 8715 tablet_bootstrap.cc:492] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Bootstrap starting.
I20260619 22:21:18.763005 8713 tablet_bootstrap.cc:654] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:18.763759 8713 log.cc:824] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:18.764011 8715 tablet_bootstrap.cc:654] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:18.764696 8715 log.cc:824] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:18.765413 8714 tablet_bootstrap.cc:492] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Bootstrap starting.
I20260619 22:21:18.765657 8713 tablet_bootstrap.cc:492] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: No bootstrap required, opened a new log
I20260619 22:21:18.765849 8713 ts_tablet_manager.cc:1397] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260619 22:21:18.766729 8715 tablet_bootstrap.cc:492] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: No bootstrap required, opened a new log
I20260619 22:21:18.766963 8715 ts_tablet_manager.cc:1397] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260619 22:21:18.767609 8714 tablet_bootstrap.cc:654] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:18.768311 8714 log.cc:824] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:18.770207 8714 tablet_bootstrap.cc:492] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: No bootstrap required, opened a new log
I20260619 22:21:18.770406 8714 ts_tablet_manager.cc:1397] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 22:21:18.771080 8713 raft_consensus.cc:348] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.771366 8713 raft_consensus.cc:374] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:18.771428 8713 raft_consensus.cc:729] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0362f0c1fc754d54be8d60b73956df39, State: Initialized, Role: FOLLOWER
I20260619 22:21:18.771907 8713 consensus_queue.cc:260] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.772321 8715 raft_consensus.cc:348] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.772562 8715 raft_consensus.cc:374] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:18.772605 8715 raft_consensus.cc:729] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247dc117e29441ee9cb12818c64ede15, State: Initialized, Role: FOLLOWER
I20260619 22:21:18.773020 8713 ts_tablet_manager.cc:1428] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Time spent starting tablet: real 0.007s user 0.008s sys 0.001s
I20260619 22:21:18.773016 8715 consensus_queue.cc:260] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.773324 8406 heartbeater.cc:503] Master 127.1.74.190:39085 was elected leader, sending a full tablet report...
I20260619 22:21:18.773998 8715 ts_tablet_manager.cc:1428] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260619 22:21:18.774354 8543 heartbeater.cc:503] Master 127.1.74.190:39085 was elected leader, sending a full tablet report...
I20260619 22:21:18.777098 8714 raft_consensus.cc:348] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.777402 8714 raft_consensus.cc:374] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:18.777472 8714 raft_consensus.cc:729] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fc676f2fb7487c93e6f8f85d30de64, State: Initialized, Role: FOLLOWER
I20260619 22:21:18.777985 8714 consensus_queue.cc:260] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.779136 8714 ts_tablet_manager.cc:1428] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Time spent starting tablet: real 0.009s user 0.002s sys 0.005s
I20260619 22:21:18.780082 8679 heartbeater.cc:503] Master 127.1.74.190:39085 was elected leader, sending a full tablet report...
W20260619 22:21:18.788228 8408 tablet.cc:2369] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:21:18.909260 8721 raft_consensus.cc:482] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:18.909585 8721 raft_consensus.cc:504] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.910878 8721 leader_election.cc:288] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505), 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109)
I20260619 22:21:18.914505 8712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.911306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:37723 (local address 127.1.74.129:40109)
0619 22:21:18.911438 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.911442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.911460 (+ 18us) server_negotiation.cc:407] Connection header received
0619 22:21:18.911533 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.911535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.911602 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.911674 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.911968 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.912475 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.913529 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.913929 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.914055 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.914108 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.914309 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.914374 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:21:18.914505 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.911201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40109 (local address 127.1.74.131:37723)
0619 22:21:18.911311 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.911329 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.911425 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.911696 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.911699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.911716 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.911851 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.911856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.912590 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.912594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.913393 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.913402 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.914052 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.914055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.914187 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.914204 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.914294 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.914369 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"spinlock_wait_cycles":7552}
I20260619 22:21:18.914683 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.911411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:50937 (local address 127.1.74.130:36505)
0619 22:21:18.911546 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.911551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.911700 (+ 149us) server_negotiation.cc:407] Connection header received
0619 22:21:18.911861 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.911864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.911920 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.912013 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:18.912252 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.912758 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.913732 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.914134 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.914264 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.914312 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.914506 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.914574 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:21:18.914703 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.911201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36505 (local address 127.1.74.131:50937)
0619 22:21:18.911606 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:18.911624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:18.911742 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:18.912021 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:18.912024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:18.912039 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:18.912140 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.912146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.912875 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.912880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.913609 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:18.913617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.914251 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.914254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:18.914360 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.914381 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:21:18.914486 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 22:21:18.914557 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":289}
I20260619 22:21:18.915297 8359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeb6b53852c4bf3b9dae1d7e8bbd7ea" candidate_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0362f0c1fc754d54be8d60b73956df39" is_pre_election: true
I20260619 22:21:18.915397 8496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeb6b53852c4bf3b9dae1d7e8bbd7ea" candidate_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247dc117e29441ee9cb12818c64ede15" is_pre_election: true
I20260619 22:21:18.915642 8359 raft_consensus.cc:2454] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6fc676f2fb7487c93e6f8f85d30de64 in term 0.
I20260619 22:21:18.915709 8496 raft_consensus.cc:2454] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6fc676f2fb7487c93e6f8f85d30de64 in term 0.
I20260619 22:21:18.916149 8567 leader_election.cc:302] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0362f0c1fc754d54be8d60b73956df39, 247dc117e29441ee9cb12818c64ede15, e6fc676f2fb7487c93e6f8f85d30de64; no voters:
I20260619 22:21:18.916436 8721 raft_consensus.cc:2781] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:18.916548 8721 raft_consensus.cc:482] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:18.916611 8721 raft_consensus.cc:3037] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:18.917723 8721 raft_consensus.cc:504] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.918143 8721 leader_election.cc:288] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [CANDIDATE]: Term 1 election: Requested vote from peers 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505), 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109)
I20260619 22:21:18.918596 8359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeb6b53852c4bf3b9dae1d7e8bbd7ea" candidate_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0362f0c1fc754d54be8d60b73956df39"
I20260619 22:21:18.918608 8496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeb6b53852c4bf3b9dae1d7e8bbd7ea" candidate_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247dc117e29441ee9cb12818c64ede15"
I20260619 22:21:18.918740 8496 raft_consensus.cc:3037] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:18.918740 8359 raft_consensus.cc:3037] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:18.920094 8359 raft_consensus.cc:2454] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6fc676f2fb7487c93e6f8f85d30de64 in term 1.
I20260619 22:21:18.920094 8496 raft_consensus.cc:2454] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6fc676f2fb7487c93e6f8f85d30de64 in term 1.
I20260619 22:21:18.920419 8567 leader_election.cc:302] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0362f0c1fc754d54be8d60b73956df39, 247dc117e29441ee9cb12818c64ede15, e6fc676f2fb7487c93e6f8f85d30de64; no voters:
I20260619 22:21:18.920591 8721 raft_consensus.cc:2781] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:18.920822 8721 raft_consensus.cc:686] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 LEADER]: Becoming Leader. State: Replica: e6fc676f2fb7487c93e6f8f85d30de64, State: Running, Role: LEADER
I20260619 22:21:18.921197 8721 consensus_queue.cc:237] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:18.925444 7441 catalog_manager.cc:5495] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131). New cstate: current_term: 1 leader_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } health_report { overall_health: HEALTHY } } }
W20260619 22:21:18.946038 8681 tablet.cc:2369] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:21:18.983592 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:18.979935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.1.74.131:33087)
0619 22:21:18.980070 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:18.980074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:18.980089 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:18.980185 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:18.980187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:18.980240 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:18.980328 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:18.980586 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.981122 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.982046 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:18.982276 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:18.982358 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:18.982671 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:18.983125 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:18.983232 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:18.983346 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 22:21:18.983400 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 22:21:18.989663 8496 raft_consensus.cc:1261] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Refusing update from remote peer e6fc676f2fb7487c93e6f8f85d30de64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:18.989692 8359 raft_consensus.cc:1261] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 1 FOLLOWER]: Refusing update from remote peer e6fc676f2fb7487c93e6f8f85d30de64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:18.990417 8721 consensus_queue.cc:1035] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:18.990660 8722 consensus_queue.cc:1035] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 22:21:18.992848 8545 tablet.cc:2369] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:21:19.000569 8726 mvcc.cc:205] Tried to move back new op lower bound from 7298693853133737984 to 7298693852864253952. Current Snapshot: MvccSnapshot[applied={T|T < 7298693853133737984}]
I20260619 22:21:19.002188 8725 mvcc.cc:205] Tried to move back new op lower bound from 7298693853133737984 to 7298693852864253952. Current Snapshot: MvccSnapshot[applied={T|T < 7298693853133737984 or (T in {7298693853133737984})}]
I20260619 22:21:19.007735 8727 mvcc.cc:205] Tried to move back new op lower bound from 7298693853133737984 to 7298693852864253952. Current Snapshot: MvccSnapshot[applied={T|T < 7298693853133737984}]
I20260619 22:21:19.009811 7441 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:19.009991 7441 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:19.015468 7441 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3b1bc7f83df4b2ea1f56bc784bed180: Sending DeleteTablet for 3 replicas of tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea
I20260619 22:21:19.016228 8476 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:19 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56998
I20260619 22:21:19.016856 8339 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:19 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:38586
I20260619 22:21:19.016978 8612 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeb6b53852c4bf3b9dae1d7e8bbd7ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:19 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:37574
I20260619 22:21:19.017469 8736 tablet_replica.cc:330] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: stopping tablet replica
I20260619 22:21:19.017822 8736 raft_consensus.cc:2229] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:19.018004 8735 tablet_replica.cc:330] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: stopping tablet replica
I20260619 22:21:19.018193 8736 raft_consensus.cc:2258] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:19.018420 8735 raft_consensus.cc:2229] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:19.018797 8735 raft_consensus.cc:2258] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:19.020035 8736 ts_tablet_manager.cc:1905] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:19.020210 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7409
I20260619 22:21:19.020218 8735 ts_tablet_manager.cc:1905] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:19.023381 8736 ts_tablet_manager.cc:1918] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:19.023617 8736 log.cc:1196] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/0eeb6b53852c4bf3b9dae1d7e8bbd7ea
I20260619 22:21:19.023983 8736 ts_tablet_manager.cc:1939] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P e6fc676f2fb7487c93e6f8f85d30de64: Deleting consensus metadata
I20260619 22:21:19.025094 8735 ts_tablet_manager.cc:1918] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:19.025259 8735 log.cc:1196] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/0eeb6b53852c4bf3b9dae1d7e8bbd7ea
I20260619 22:21:19.025645 8735 ts_tablet_manager.cc:1939] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 0362f0c1fc754d54be8d60b73956df39: Deleting consensus metadata
W20260619 22:21:19.027395 8568 connection.cc:535] server connection from 127.0.0.1:37574 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 22:21:19.027428 8294 connection.cc:535] server connection from 127.0.0.1:38586 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 22:21:19.028779 8734 tablet_replica.cc:330] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: stopping tablet replica
I20260619 22:21:19.029150 8734 raft_consensus.cc:2229] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:19.029481 8734 raft_consensus.cc:2258] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:19.031114 8734 ts_tablet_manager.cc:1905] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:19.034300 8734 ts_tablet_manager.cc:1918] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:19.034408 8734 log.cc:1196] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/0eeb6b53852c4bf3b9dae1d7e8bbd7ea
I20260619 22:21:19.034739 8734 ts_tablet_manager.cc:1939] T 0eeb6b53852c4bf3b9dae1d7e8bbd7ea P 247dc117e29441ee9cb12818c64ede15: Deleting consensus metadata
W20260619 22:21:19.035959 8430 connection.cc:441] server connection from 127.0.0.1:56998 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:56998 (request call id 1) could send its response
I20260619 22:21:19.040802 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:19.031493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.1.74.189:34295)
0619 22:21:19.031620 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:19.031625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:19.031688 (+ 63us) server_negotiation.cc:407] Connection header received
0619 22:21:19.031918 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:19.031921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:19.031973 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:19.032028 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:19.032971 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.033736 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.035149 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.035455 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.035531 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:19.037001 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:19.037005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:19.037007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:19.037037 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:19.038942 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.039424 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.039429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.039430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.039513 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.039829 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.039833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.039834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.040050 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:19.040152 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:19.040508 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 22:21:19.040639 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":896}
I20260619 22:21:19.040802 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:19.031024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.1.74.188:34871)
0619 22:21:19.031173 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:19.031177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:19.031191 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:19.031231 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:19.031235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:19.031282 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:19.031354 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:19.032614 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.033428 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.034631 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.035037 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.035111 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:19.037071 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:19.037076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:19.037081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:19.037108 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:19.039056 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.039558 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.039562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.039567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.039647 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.039948 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.039951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.039952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.040159 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:19.040248 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:19.040542 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 22:21:19.040654 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:21:19.043783 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:19.018297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.1.74.189:34295)
0619 22:21:19.018423 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:19.018428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:19.028536 (+ 10108us) server_negotiation.cc:407] Connection header received
0619 22:21:19.028668 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:19.028671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:19.028731 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:19.028809 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:19.036202 (+ 7393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.037017 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.038336 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.038689 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.038781 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:19.040381 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:19.040384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:19.040387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:19.040414 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:19.042011 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.042502 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.042506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.042508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.042599 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.042914 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.042917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.042919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.043106 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:19.043215 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:19.043535 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 22:21:19.043631 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 22:21:19.051806 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:19.018408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59598 (local address 127.1.74.188:34871)
0619 22:21:19.018543 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:19.018547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:19.043858 (+ 25311us) server_negotiation.cc:407] Connection header received
0619 22:21:19.044020 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:19.044023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:19.044067 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:19.044115 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:19.044913 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.045472 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.046570 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:19.046940 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:19.047011 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:19.048794 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:19.048797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:19.048799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:19.048819 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:19.050119 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.050565 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.050567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.050568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.050662 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:19.051031 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:19.051034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:19.051036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:19.051201 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:19.051294 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:19.051552 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 22:21:19.051638 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:21:20.003266 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.002694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:55193)
0619 22:21:20.003019 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.003130 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":83,"threads_started":1}
W20260619 22:21:20.003599 8543 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:39085 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.005405 8745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.004994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:46645)
0619 22:21:20.005225 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.005322 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
W20260619 22:21:20.005790 8679 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:39085 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.010151 8746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.009664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:57205)
0619 22:21:20.009929 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.010029 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
W20260619 22:21:20.010592 8406 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:39085 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.515549 8747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c3b1bc7f83df4b2ea1f56bc784bed180)
I20260619 22:21:20.515782 8747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } }
I20260619 22:21:20.516763 8747 leader_election.cc:288] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085), 372fc71d57c940e3bacacd31cf83db4f (127.1.74.189:34295)
I20260619 22:21:20.517506 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.516972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55714)
0619 22:21:20.517277 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.517370 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":23,"thread_start_us":72,"threads_started":1}
I20260619 22:21:20.517527 8751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c3b1bc7f83df4b2ea1f56bc784bed180)
I20260619 22:21:20.517634 8751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } }
W20260619 22:21:20.517696 8212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.518574 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.518327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55738)
0619 22:21:20.518459 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.518512 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 22:21:20.518590 8753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.518226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55730)
0619 22:21:20.518436 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.518507 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":57,"threads_started":1}
W20260619 22:21:20.518781 8028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
W20260619 22:21:20.518878 8212 leader_election.cc:334] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085): Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.519106 8751 leader_election.cc:288] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085), 9990ef2bf21245a49dbb64980e05a80a (127.1.74.188:34871)
I20260619 22:21:20.519622 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.519382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55742)
0619 22:21:20.519481 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.519546 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 22:21:20.519893 8028 leader_election.cc:334] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085): Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.521380 8750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.517136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.1.74.189:34295)
0619 22:21:20.517485 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.517490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.517512 (+ 22us) server_negotiation.cc:407] Connection header received
0619 22:21:20.517715 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.517718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.517789 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.517876 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.518117 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.518952 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.520416 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.520913 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.520994 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.521041 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.521247 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.521301 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":60,"threads_started":1}
I20260619 22:21:20.521420 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.516972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:34295 (local address 127.0.0.1:42068)
0619 22:21:20.517252 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.517268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.517384 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.517888 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.517891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.517908 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.518009 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.518017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.519132 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.519137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.520303 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.520312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.521033 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.521036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.521126 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.521149 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.521227 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.521293 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260619 22:21:20.521955 8052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9990ef2bf21245a49dbb64980e05a80a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "372fc71d57c940e3bacacd31cf83db4f" is_pre_election: true
I20260619 22:21:20.522166 8052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9990ef2bf21245a49dbb64980e05a80a in term 1.
I20260619 22:21:20.522509 8213 leader_election.cc:302] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 372fc71d57c940e3bacacd31cf83db4f, 9990ef2bf21245a49dbb64980e05a80a; no voters: c3b1bc7f83df4b2ea1f56bc784bed180
I20260619 22:21:20.522784 8753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.518574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:34871 (local address 127.0.0.1:59624)
0619 22:21:20.518726 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.518739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.518822 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.519594 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.519599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.519621 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.519703 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.519708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.520558 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.520560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.521589 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.521599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.522386 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.522393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.522492 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.522509 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.522583 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.522668 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260619 22:21:20.522789 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.519126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59624 (local address 127.1.74.188:34871)
0619 22:21:20.519339 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.519349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.519363 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:20.519399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.519401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.519444 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.519558 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.519844 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.520454 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.521707 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.522270 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.522383 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.522440 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.522621 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.522684 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
I20260619 22:21:20.522930 8747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 22:21:20.522977 8747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Starting leader election (detected failure of leader c3b1bc7f83df4b2ea1f56bc784bed180)
I20260619 22:21:20.523052 8747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:21:20.523459 8236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "372fc71d57c940e3bacacd31cf83db4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9990ef2bf21245a49dbb64980e05a80a" is_pre_election: true
I20260619 22:21:20.524271 8747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } }
I20260619 22:21:20.524530 8236 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 372fc71d57c940e3bacacd31cf83db4f in current term 2: Already voted for candidate 9990ef2bf21245a49dbb64980e05a80a in this term.
I20260619 22:21:20.524669 8747 leader_election.cc:288] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [CANDIDATE]: Term 2 election: Requested vote from peers c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085), 372fc71d57c940e3bacacd31cf83db4f (127.1.74.189:34295)
I20260619 22:21:20.525034 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.524808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55758)
0619 22:21:20.524895 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.524962 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 22:21:20.525117 8052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9990ef2bf21245a49dbb64980e05a80a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "372fc71d57c940e3bacacd31cf83db4f"
I20260619 22:21:20.525130 8028 leader_election.cc:302] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 372fc71d57c940e3bacacd31cf83db4f; no voters: 9990ef2bf21245a49dbb64980e05a80a, c3b1bc7f83df4b2ea1f56bc784bed180
I20260619 22:21:20.525238 8052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:21:20.525701 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.525497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55762)
0619 22:21:20.525604 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.525651 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 22:21:20.525863 8212 leader_election.cc:334] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085): Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
I20260619 22:21:20.526248 8052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9990ef2bf21245a49dbb64980e05a80a in term 2.
I20260619 22:21:20.526361 8751 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 22:21:20.526522 8213 leader_election.cc:302] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 372fc71d57c940e3bacacd31cf83db4f, 9990ef2bf21245a49dbb64980e05a80a; no voters: c3b1bc7f83df4b2ea1f56bc784bed180
I20260619 22:21:20.526748 8747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 2 FOLLOWER]: Leader election won for term 2
I20260619 22:21:20.526983 8747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [term 2 LEADER]: Becoming Leader. State: Replica: 9990ef2bf21245a49dbb64980e05a80a, State: Running, Role: LEADER
I20260619 22:21:20.527230 8747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } }
I20260619 22:21:20.528498 8757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9990ef2bf21245a49dbb64980e05a80a. Latest consensus state: current_term: 2 leader_uuid: "9990ef2bf21245a49dbb64980e05a80a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:20.528626 8757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:20.528895 8759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:20.531025 8759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:20.531405 8759 catalog_manager.cc:1259] Loaded cluster ID: 16beac010ee74b349c1a6c813c8158e9
I20260619 22:21:20.531472 8759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:20.531982 8759 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:20.532364 8759 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Loaded TSK: 0
I20260619 22:21:20.533008 8759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:20.558820 8225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:20.562131 8225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:20.565261 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.564977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55786)
0619 22:21:20.565120 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.565181 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 22:21:20.565260 8052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [term 2 FOLLOWER]: Refusing update from remote peer 9990ef2bf21245a49dbb64980e05a80a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 22:21:20.565837 8757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 22:21:20.565976 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.565760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.0.0.1:55794)
0619 22:21:20.565868 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.565918 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 22:21:20.566085 8212 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a -> Peer c3b1bc7f83df4b2ea1f56bc784bed180 (127.1.74.190:39085): Couldn't send request to peer c3b1bc7f83df4b2ea1f56bc784bed180. Status: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 22:21:20.567369 8751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9990ef2bf21245a49dbb64980e05a80a. Latest consensus state: current_term: 2 leader_uuid: "9990ef2bf21245a49dbb64980e05a80a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:20.567516 8751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:20.568544 8747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9990ef2bf21245a49dbb64980e05a80a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:20.568697 8747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:20.568904 8751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9990ef2bf21245a49dbb64980e05a80a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c3b1bc7f83df4b2ea1f56bc784bed180" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39085 } } peers { permanent_uuid: "372fc71d57c940e3bacacd31cf83db4f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 34295 } attrs { promote: false } } peers { permanent_uuid: "9990ef2bf21245a49dbb64980e05a80a" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 34871 } attrs { promote: false } } }
I20260619 22:21:20.569020 8751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 372fc71d57c940e3bacacd31cf83db4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:21:20.579121 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.574583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.1.74.129:40109)
0619 22:21:20.574817 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.574821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.574833 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:20.574925 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.574928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.574990 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.575092 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.575663 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.576450 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.577675 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.578315 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.578425 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.578502 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.578816 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.578944 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260619 22:21:20.579392 8766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.575198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.1.74.130:36505)
0619 22:21:20.575477 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.575483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.575766 (+ 283us) server_negotiation.cc:407] Connection header received
0619 22:21:20.575924 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.575927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.575976 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.576060 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.576384 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.576884 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.578126 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.578716 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.578814 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.578909 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.579196 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.579270 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":98,"threads_started":1}
I20260619 22:21:20.579382 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.575083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36505 (local address 127.0.0.1:57006)
0619 22:21:20.575688 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.575703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.575826 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.576169 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.576172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.576185 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.576283 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.576289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.576995 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.577001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.578004 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.578011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.578968 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.578974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.579078 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.579103 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.579179 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.579250 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":19}
I20260619 22:21:20.580060 8476 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef47a3537814d0c9c5edcf26a49b9e2 (DEFAULT_TABLE table=test-table [id=5473697a508446e2823aae32db756a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:20.580145 8749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.574458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40109 (local address 127.0.0.1:38594)
0619 22:21:20.574623 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.574641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.574724 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.575106 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.575112 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.575386 (+ 274us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.575491 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.575500 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.576581 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.576583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.577535 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.577545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.578562 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.578566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.578657 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.578681 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.579815 (+ 1134us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.579898 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":46}
I20260619 22:21:20.580400 8476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef47a3537814d0c9c5edcf26a49b9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:20.581574 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.575295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:33087 (local address 127.0.0.1:37602)
0619 22:21:20.577264 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.577277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.577404 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.577787 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.577790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.577808 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.577897 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.577903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.578894 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.578898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.580287 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.580298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.581141 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.581145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.581237 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.581260 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.581369 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.581443 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1915,"thread_start_us":113,"threads_started":1}
I20260619 22:21:20.581609 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.575431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.1.74.131:33087)
0619 22:21:20.575679 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.575684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.577372 (+ 1688us) server_negotiation.cc:407] Connection header received
0619 22:21:20.577506 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.577509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.577568 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.577670 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.578032 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.578737 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.580502 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.581009 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.581144 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.581213 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.581387 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.581463 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
I20260619 22:21:20.582311 8612 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef47a3537814d0c9c5edcf26a49b9e2 (DEFAULT_TABLE table=test-table [id=5473697a508446e2823aae32db756a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:20.582607 8612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef47a3537814d0c9c5edcf26a49b9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:20.583269 8339 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef47a3537814d0c9c5edcf26a49b9e2 (DEFAULT_TABLE table=test-table [id=5473697a508446e2823aae32db756a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:20.584070 8339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef47a3537814d0c9c5edcf26a49b9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:20.584681 8769 tablet_bootstrap.cc:492] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Bootstrap starting.
I20260619 22:21:20.585328 8770 tablet_bootstrap.cc:492] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Bootstrap starting.
I20260619 22:21:20.586314 8769 tablet_bootstrap.cc:654] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:20.587746 8769 tablet_bootstrap.cc:492] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: No bootstrap required, opened a new log
I20260619 22:21:20.587841 8769 ts_tablet_manager.cc:1397] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 22:21:20.588362 8772 tablet_bootstrap.cc:492] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39: Bootstrap starting.
I20260619 22:21:20.588328 8769 raft_consensus.cc:348] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.588442 8769 raft_consensus.cc:374] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:20.588477 8769 raft_consensus.cc:729] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247dc117e29441ee9cb12818c64ede15, State: Initialized, Role: FOLLOWER
I20260619 22:21:20.588611 8769 consensus_queue.cc:260] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.588955 8769 ts_tablet_manager.cc:1428] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260619 22:21:20.589757 8772 tablet_bootstrap.cc:654] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:20.591128 8772 tablet_bootstrap.cc:492] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39: No bootstrap required, opened a new log
I20260619 22:21:20.591187 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.589475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:51351)
0619 22:21:20.591015 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.591073 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1470,"thread_start_us":684,"threads_started":1}
I20260619 22:21:20.591219 8772 ts_tablet_manager.cc:1397] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 22:21:20.591665 8772 raft_consensus.cc:348] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.591789 8772 raft_consensus.cc:374] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:20.591842 8772 raft_consensus.cc:729] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0362f0c1fc754d54be8d60b73956df39, State: Initialized, Role: FOLLOWER
I20260619 22:21:20.591997 8772 consensus_queue.cc:260] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.592367 8772 ts_tablet_manager.cc:1428] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:20.592607 8770 tablet_bootstrap.cc:654] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:20.593308 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.592873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:36889)
0619 22:21:20.593112 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.593179 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
I20260619 22:21:20.593295 8542 heartbeater.cc:503] Master 127.1.74.188:34871 was elected leader, sending a full tablet report...
I20260619 22:21:20.594507 8405 heartbeater.cc:503] Master 127.1.74.188:34871 was elected leader, sending a full tablet report...
I20260619 22:21:20.594769 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.593473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:45397)
0619 22:21:20.594649 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.594705 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1108}
I20260619 22:21:20.595772 8770 tablet_bootstrap.cc:492] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: No bootstrap required, opened a new log
I20260619 22:21:20.595588 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.593804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:36201)
0619 22:21:20.595458 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.595520 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1575}
I20260619 22:21:20.595907 8770 ts_tablet_manager.cc:1397] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260619 22:21:20.596275 8770 raft_consensus.cc:348] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.596391 8770 raft_consensus.cc:374] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:20.596436 8770 raft_consensus.cc:729] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fc676f2fb7487c93e6f8f85d30de64, State: Initialized, Role: FOLLOWER
I20260619 22:21:20.596570 8770 consensus_queue.cc:260] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.596871 8770 ts_tablet_manager.cc:1428] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:20.598119 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.597207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:59681)
0619 22:21:20.597789 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.597884 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260619 22:21:20.598407 8678 heartbeater.cc:503] Master 127.1.74.188:34871 was elected leader, sending a full tablet report...
I20260619 22:21:20.598883 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.598649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:38537)
0619 22:21:20.598732 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.598801 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 22:21:20.697741 8776 raft_consensus.cc:482] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:20.697922 8776 raft_consensus.cc:504] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.699167 8776 leader_election.cc:288] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505), e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087)
I20260619 22:21:20.702909 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.699484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36505 (local address 127.1.74.129:45225)
0619 22:21:20.699611 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.699628 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.699728 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.700068 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.700072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.700096 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.700235 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.700241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.700984 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.700987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.701816 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.701824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.702457 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.702460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.702574 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.702599 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.702681 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.702754 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":20}
I20260619 22:21:20.702908 8766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.699543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:45225 (local address 127.1.74.130:36505)
0619 22:21:20.699684 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.699690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.699707 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:21:20.699848 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.699852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.699906 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.699980 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.700350 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.700855 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.701939 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.702340 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.702467 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.702524 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.702709 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.702776 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:21:20.703347 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.699888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:42475 (local address 127.1.74.131:33087)
0619 22:21:20.700005 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.700009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.700179 (+ 170us) server_negotiation.cc:407] Connection header received
0619 22:21:20.700371 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.700375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.700428 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.700467 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:20.700783 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.701288 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.702326 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.702711 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.702829 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.702905 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.703142 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.703252 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 22:21:20.703321 8496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef47a3537814d0c9c5edcf26a49b9e2" candidate_uuid: "0362f0c1fc754d54be8d60b73956df39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247dc117e29441ee9cb12818c64ede15" is_pre_election: true
I20260619 22:21:20.703483 8496 raft_consensus.cc:2454] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0362f0c1fc754d54be8d60b73956df39 in term 0.
I20260619 22:21:20.703763 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.699748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:33087 (local address 127.1.74.129:42475)
0619 22:21:20.700073 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.700093 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:20.700251 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:20.700585 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:20.700587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:20.700600 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:20.700682 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.700685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.701397 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.701402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.702188 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:20.702196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.702830 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.702833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:20.702981 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.703014 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 22:21:20.703508 (+ 494us) client_negotiation.cc:240] Negotiation successful
0619 22:21:20.703594 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":25,"thread_start_us":124,"threads_started":1}
I20260619 22:21:20.703852 8292 leader_election.cc:302] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0362f0c1fc754d54be8d60b73956df39, 247dc117e29441ee9cb12818c64ede15; no voters:
I20260619 22:21:20.704079 8776 raft_consensus.cc:2781] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:20.704154 8776 raft_consensus.cc:482] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:20.704198 8776 raft_consensus.cc:3037] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:20.704463 8632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef47a3537814d0c9c5edcf26a49b9e2" candidate_uuid: "0362f0c1fc754d54be8d60b73956df39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" is_pre_election: true
I20260619 22:21:20.704708 8632 raft_consensus.cc:2454] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0362f0c1fc754d54be8d60b73956df39 in term 0.
I20260619 22:21:20.705464 8776 raft_consensus.cc:504] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.705799 8776 leader_election.cc:288] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [CANDIDATE]: Term 1 election: Requested vote from peers 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505), e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087)
I20260619 22:21:20.706120 8496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef47a3537814d0c9c5edcf26a49b9e2" candidate_uuid: "0362f0c1fc754d54be8d60b73956df39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "247dc117e29441ee9cb12818c64ede15"
I20260619 22:21:20.706212 8496 raft_consensus.cc:3037] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:20.706230 8632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef47a3537814d0c9c5edcf26a49b9e2" candidate_uuid: "0362f0c1fc754d54be8d60b73956df39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6fc676f2fb7487c93e6f8f85d30de64"
I20260619 22:21:20.706353 8632 raft_consensus.cc:3037] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:20.707221 8496 raft_consensus.cc:2454] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0362f0c1fc754d54be8d60b73956df39 in term 1.
I20260619 22:21:20.707424 8632 raft_consensus.cc:2454] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0362f0c1fc754d54be8d60b73956df39 in term 1.
I20260619 22:21:20.707506 8292 leader_election.cc:302] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0362f0c1fc754d54be8d60b73956df39, 247dc117e29441ee9cb12818c64ede15; no voters:
I20260619 22:21:20.707665 8776 raft_consensus.cc:2781] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:20.707890 8776 raft_consensus.cc:686] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 1 LEADER]: Becoming Leader. State: Replica: 0362f0c1fc754d54be8d60b73956df39, State: Running, Role: LEADER
I20260619 22:21:20.708142 8776 consensus_queue.cc:237] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } }
I20260619 22:21:20.708596 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.708361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:42863)
0619 22:21:20.708475 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.708530 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 22:21:20.710451 8226 catalog_manager.cc:5495] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "0362f0c1fc754d54be8d60b73956df39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0362f0c1fc754d54be8d60b73956df39" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 } health_report { overall_health: UNKNOWN } } }
I20260619 22:21:20.738056 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.734587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.1.74.129:40109)
0619 22:21:20.734740 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:20.734747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:20.734764 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:21:20.734902 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:20.734907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:20.734970 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:20.735095 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:20.735381 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.735892 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.736741 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:20.736962 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:20.737041 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:20.737266 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:20.737715 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:20.737806 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:20.737892 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 22:21:20.737944 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:21:20.741138 8496 raft_consensus.cc:1261] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Refusing update from remote peer 0362f0c1fc754d54be8d60b73956df39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:20.741286 8632 raft_consensus.cc:1261] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Refusing update from remote peer 0362f0c1fc754d54be8d60b73956df39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:20.741740 8782 consensus_queue.cc:1035] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "247dc117e29441ee9cb12818c64ede15" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:20.741943 8776 consensus_queue.cc:1035] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6fc676f2fb7487c93e6f8f85d30de64" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 33087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:20.743913 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.743405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:35721)
0619 22:21:20.743764 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.743824 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277}
I20260619 22:21:20.744094 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.743812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:45493)
0619 22:21:20.743963 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.744015 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 22:21:20.747560 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.747285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:38033)
0619 22:21:20.747419 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.747483 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 22:21:20.747697 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.747214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.130:57503)
0619 22:21:20.747565 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.747617 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
I20260619 22:21:20.749541 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.749319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.129:59895)
0619 22:21:20.749422 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.749473 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 22:21:20.751294 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.751030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39085 (local address 127.1.74.131:56195)
0619 22:21:20.751177 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.751227 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 22:21:20.756858 8225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:20.757050 8225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:20.761076 8225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9990ef2bf21245a49dbb64980e05a80a: Sending DeleteTablet for 3 replicas of tablet 8ef47a3537814d0c9c5edcf26a49b9e2
I20260619 22:21:20.761983 8339 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef47a3537814d0c9c5edcf26a49b9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:20 UTC) from {username='oryx', principal='oryx/127.1.74.188@KRBTEST.COM'} at 127.0.0.1:38594
I20260619 22:21:20.762194 8476 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef47a3537814d0c9c5edcf26a49b9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:20 UTC) from {username='oryx', principal='oryx/127.1.74.188@KRBTEST.COM'} at 127.0.0.1:57006
I20260619 22:21:20.762194 8612 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef47a3537814d0c9c5edcf26a49b9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:20 UTC) from {username='oryx', principal='oryx/127.1.74.188@KRBTEST.COM'} at 127.0.0.1:37602
I20260619 22:21:20.762596 8798 tablet_replica.cc:330] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: stopping tablet replica
I20260619 22:21:20.762734 8798 raft_consensus.cc:2229] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:20.762831 8796 tablet_replica.cc:330] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39: stopping tablet replica
I20260619 22:21:20.762908 8798 raft_consensus.cc:2258] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:20.763064 8796 raft_consensus.cc:2229] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 0362f0c1fc754d54be8d60b73956df39 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:20.763307 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 8186
I20260619 22:21:20.763664 8798 ts_tablet_manager.cc:1905] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:20.764024 8797 tablet_replica.cc:330] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: stopping tablet replica
I20260619 22:21:20.764168 8797 raft_consensus.cc:2229] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:20.764300 8797 raft_consensus.cc:2258] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:20.765039 8797 ts_tablet_manager.cc:1905] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:20.766523 8798 ts_tablet_manager.cc:1918] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:20.766604 8798 log.cc:1196] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/8ef47a3537814d0c9c5edcf26a49b9e2
I20260619 22:21:20.766924 8798 ts_tablet_manager.cc:1939] T 8ef47a3537814d0c9c5edcf26a49b9e2 P 247dc117e29441ee9cb12818c64ede15: Deleting consensus metadata
I20260619 22:21:20.767756 8210 catalog_manager.cc:4841] TS 247dc117e29441ee9cb12818c64ede15 (127.1.74.130:36505): tablet 8ef47a3537814d0c9c5edcf26a49b9e2 (table test-table [id=5473697a508446e2823aae32db756a72]) successfully deleted
I20260619 22:21:20.767827 8797 ts_tablet_manager.cc:1918] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:20.767896 8797 log.cc:1196] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/8ef47a3537814d0c9c5edcf26a49b9e2
I20260619 22:21:20.768085 8797 ts_tablet_manager.cc:1939] T 8ef47a3537814d0c9c5edcf26a49b9e2 P e6fc676f2fb7487c93e6f8f85d30de64: Deleting consensus metadata
I20260619 22:21:20.768731 8210 catalog_manager.cc:4841] TS e6fc676f2fb7487c93e6f8f85d30de64 (127.1.74.131:33087): tablet 8ef47a3537814d0c9c5edcf26a49b9e2 (table test-table [id=5473697a508446e2823aae32db756a72]) successfully deleted
I20260619 22:21:20.771180 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 8413
I20260619 22:21:20.771534 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:20.771207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40109 (local address 127.0.0.1:38602)
0619 22:21:20.771364 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:20.771430 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 22:21:20.772156 8212 catalog_manager.cc:4584] TS 0362f0c1fc754d54be8d60b73956df39 (127.1.74.129:40109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ef47a3537814d0c9c5edcf26a49b9e2: Network error: Client connection negotiation failed: client connection to 127.1.74.129:40109: connect: Connection refused (error 111)
I20260619 22:21:20.777395 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 8549
I20260619 22:21:20.784181 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 7905
I20260619 22:21:20.789988 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 8014
2026-06-19T22:21:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5490 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:20 dist-test-slave-k84h krb5kdc[8801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:20 dist-test-slave-k84h krb5kdc[8801](info): set up 2 sockets
Jun 19 22:21:20 dist-test-slave-k84h krb5kdc[8801](info): commencing operation
krb5kdc: starting...
W20260619 22:21:21.835585 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:21 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab.
Entry for principal oryx/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab.
I20260619 22:21:21.962569 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42301
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab
--principal=oryx/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:42301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:22.072328 8814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:22.072615 8814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:22.072655 8814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:22.076702 8814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:22.076772 8814 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 22:21:22.076790 8814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:22.076808 8814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:22.076824 8814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:22.076840 8814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:22.082163 8814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:42301
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.190.keytab
--principal=oryx/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:22.083379 8814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:22.084416 8814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:22.093286 8820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.093299 8822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.093294 8819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:22.093355 8814 server_base.cc:1034] running on GCE node
I20260619 22:21:22.094117 8814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:22.095415 8814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:22.096632 8814 hybrid_clock.cc:648] HybridClock initialized: now 1781907682096604 us; error 44 us; skew 500 ppm
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:22.099689 8814 init.cc:377] Logged in from keytab as oryx/127.1.74.190@KRBTEST.COM (short username oryx)
I20260619 22:21:22.100642 8814 webserver.cc:460] Webserver started at http://127.1.74.190:38143/ using document root <none> and password file <none>
I20260619 22:21:22.101186 8814 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:22.101251 8814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:22.101420 8814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:22.103583 8814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "f5b03c247f6643be8d7042d1bec6caac"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "95c48529fc7147d3b9ce688f51448fb7"
server_key_iv: "c0abdb4c00a78873997bbbb302c36887"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.104108 8814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "f5b03c247f6643be8d7042d1bec6caac"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "95c48529fc7147d3b9ce688f51448fb7"
server_key_iv: "c0abdb4c00a78873997bbbb302c36887"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.107987 8814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:21:22.110620 8829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.112159 8814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 22:21:22.112298 8814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "f5b03c247f6643be8d7042d1bec6caac"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "95c48529fc7147d3b9ce688f51448fb7"
server_key_iv: "c0abdb4c00a78873997bbbb302c36887"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.112427 8814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:22.136639 8814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:22.140136 8814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:22.140340 8814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:22.148409 8814 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:42301
I20260619 22:21:22.148432 8891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:42301 every 8 connection(s)
I20260619 22:21:22.149374 8814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:22.152072 8892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:22.157706 8892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Bootstrap starting.
I20260619 22:21:22.159431 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 8814
I20260619 22:21:22.159555 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:22.159826 1322 external_mini_cluster.cc:1428] Setting key bfeeaf03d65b6df993e442a57b6ea59d
I20260619 22:21:22.160920 8892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:22.161857 8892 log.cc:824] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:22.164099 8892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: No bootstrap required, opened a new log
I20260619 22:21:22.168290 8892 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b03c247f6643be8d7042d1bec6caac" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42301 } }
I20260619 22:21:22.168517 8892 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:22.168551 8892 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b03c247f6643be8d7042d1bec6caac, State: Initialized, Role: FOLLOWER
I20260619 22:21:22.169036 8892 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b03c247f6643be8d7042d1bec6caac" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42301 } }
I20260619 22:21:22.169160 8892 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:22.169246 8892 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:22.169348 8892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:22.170583 8892 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b03c247f6643be8d7042d1bec6caac" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42301 } }
I20260619 22:21:22.171065 8892 leader_election.cc:302] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5b03c247f6643be8d7042d1bec6caac; no voters:
I20260619 22:21:22.171465 8892 leader_election.cc:288] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:22.171586 8897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:22.171833 8897 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [term 1 LEADER]: Becoming Leader. State: Replica: f5b03c247f6643be8d7042d1bec6caac, State: Running, Role: LEADER
I20260619 22:21:22.172176 8897 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b03c247f6643be8d7042d1bec6caac" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42301 } }
I20260619 22:21:22.172902 8892 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:22.174444 8899 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5b03c247f6643be8d7042d1bec6caac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b03c247f6643be8d7042d1bec6caac" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42301 } } }
I20260619 22:21:22.174487 8898 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5b03c247f6643be8d7042d1bec6caac. Latest consensus state: current_term: 1 leader_uuid: "f5b03c247f6643be8d7042d1bec6caac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5b03c247f6643be8d7042d1bec6caac" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42301 } } }
I20260619 22:21:22.174587 8899 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:22.174620 8898 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:22.174990 8909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:22.175879 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.160928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.1.74.190:42301)
0619 22:21:22.161404 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.161423 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.161464 (+ 41us) server_negotiation.cc:407] Connection header received
0619 22:21:22.162355 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.162375 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.162765 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.163168 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.163791 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.165213 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.166111 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.166878 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.167080 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.169865 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.169871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.169890 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.169922 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.172023 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.172560 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.172565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.172572 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.172713 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.173049 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.173052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.173054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.173588 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.173783 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.174498 (+ 715us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.174825 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":160,"threads_started":1}
I20260619 22:21:22.179064 8909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:22.184448 8909 catalog_manager.cc:1347] Generated new cluster ID: da73dfd03c174312b9ecee224073b64c
I20260619 22:21:22.184522 8909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:22.210029 8909 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:22.211320 8909 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:22.219643 8909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Generated new TSK 0
I20260619 22:21:22.220568 8909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab.
Entry for principal oryx/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab.
I20260619 22:21:22.253703 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42301
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:22.369437 8918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:22.369743 8918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:22.369822 8918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:22.373862 8918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:22.373942 8918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:22.374030 8918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:21:22.374101 8918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:22.379256 8918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:42301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:22.380295 8918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:22.381294 8918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:22.391268 8923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.391276 8924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.391276 8926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:22.391630 8918 server_base.cc:1034] running on GCE node
I20260619 22:21:22.392620 8918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:22.393265 8918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:22.394456 8918 hybrid_clock.cc:648] HybridClock initialized: now 1781907682394441 us; error 35 us; skew 500 ppm
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:22.397256 8918 init.cc:377] Logged in from keytab as oryx/127.1.74.129@KRBTEST.COM (short username oryx)
I20260619 22:21:22.398144 8918 webserver.cc:460] Webserver started at http://127.1.74.129:42187/ using document root <none> and password file <none>
I20260619 22:21:22.398692 8918 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:22.398784 8918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:22.399030 8918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:22.400988 8918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "ed51c42758a94de0b08bce0a6ac860e8"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "7b2676b95139e4ab6050c85b1abc62b3"
server_key_iv: "c56dfba4ef4a2c3dad3a940401b062e1"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.401525 8918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "ed51c42758a94de0b08bce0a6ac860e8"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "7b2676b95139e4ab6050c85b1abc62b3"
server_key_iv: "c56dfba4ef4a2c3dad3a940401b062e1"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.405133 8918 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 22:21:22.407456 8933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.408612 8918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:22.408741 8918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "ed51c42758a94de0b08bce0a6ac860e8"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "7b2676b95139e4ab6050c85b1abc62b3"
server_key_iv: "c56dfba4ef4a2c3dad3a940401b062e1"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.408846 8918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:22.431571 8918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:22.434607 8918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:22.434829 8918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:22.436404 8918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:22.436476 8918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.436554 8918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:22.436599 8918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:22.447695 8918 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:39365
I20260619 22:21:22.447762 9048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:39365 every 8 connection(s)
I20260619 22:21:22.448774 8918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:22.450201 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.437248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:40187 (local address 127.1.74.190:42301)
0619 22:21:22.437431 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.437438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.438273 (+ 835us) server_negotiation.cc:407] Connection header received
0619 22:21:22.439324 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.439328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.439386 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.439472 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.440769 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.441414 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.442690 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.442990 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.443106 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.446331 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.446335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.446338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.446369 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.448051 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.448616 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.448619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.448620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.448691 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.449039 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.449043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.449044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.449287 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.449380 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.449961 (+ 581us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.450081 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 22:21:22.451045 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 8918
I20260619 22:21:22.451115 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.437637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.1.74.129:40187)
0619 22:21:22.438091 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.438143 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.439107 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.439640 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.439649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.440025 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:22.440594 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.440643 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.441544 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.441547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.442572 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.442582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.443144 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.443149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.443319 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.444024 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:22.444053 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:22.446157 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:22.448166 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.448174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.448187 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.448516 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.448781 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.448788 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.448793 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.448950 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.449431 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:22.449437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:22.449712 (+ 275us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.449869 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.450113 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":133,"threads_started":1}
I20260619 22:21:22.451185 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:22.451514 1322 external_mini_cluster.cc:1428] Setting key 510c5c937b13ce814a7ae27130964899
I20260619 22:21:22.452493 9049 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42301
I20260619 22:21:22.452662 9049 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:22.453282 9049 heartbeater.cc:511] Master 127.1.74.190:42301 requested a full tablet report, sending...
I20260619 22:21:22.454958 8845 ts_manager.cc:194] Registered new tserver with Master: ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365)
I20260619 22:21:22.456647 8845 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:40187
WARNING: no policy specified for oryx/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:22.462625 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.454812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.1.74.129:42921)
0619 22:21:22.455171 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.455280 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.455428 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.455695 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.455698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.455904 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:22.455998 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.456003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.456794 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.456797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.457502 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.457514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.458171 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.458174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.458286 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.458963 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:22.458977 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:22.459387 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:22.461047 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.461052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.461054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.461254 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.461577 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.461579 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.461580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.461654 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.462029 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:22.462033 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:22.462139 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.462246 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.462406 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260619 22:21:22.462625 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.455157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:42921 (local address 127.1.74.190:42301)
0619 22:21:22.455255 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.455258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.455361 (+ 103us) server_negotiation.cc:407] Connection header received
0619 22:21:22.455540 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.455544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.455586 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.455679 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.456115 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.456624 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.457643 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.457939 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.458018 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.459524 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.459528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.459530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.459554 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.460932 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.461353 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.461358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.461359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.461463 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.461750 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.461755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.461756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.461935 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.462042 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.462318 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.462442 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":28}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab.
Entry for principal oryx/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab.
I20260619 22:21:22.481775 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42301
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:22.598178 9058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:22.598519 9058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:22.598603 9058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:22.602700 9058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:22.602792 9058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:22.602880 9058 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:21:22.602964 9058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:22.608285 9058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:42301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:22.609403 9058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:22.610445 9058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:22.620232 9064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.620229 9063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.620275 9066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:22.620461 9058 server_base.cc:1034] running on GCE node
I20260619 22:21:22.620898 9058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:22.621595 9058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:22.622797 9058 hybrid_clock.cc:648] HybridClock initialized: now 1781907682622760 us; error 48 us; skew 500 ppm
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:22.625681 9058 init.cc:377] Logged in from keytab as oryx/127.1.74.130@KRBTEST.COM (short username oryx)
I20260619 22:21:22.626583 9058 webserver.cc:460] Webserver started at http://127.1.74.130:44983/ using document root <none> and password file <none>
I20260619 22:21:22.627199 9058 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:22.627265 9058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:22.627524 9058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:22.629565 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "f7e295e176ea4f86a3045772ca8c753c"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "76a88ee90a602faebe5e8c75f4900f03"
server_key_iv: "ad2f3c4c597fe85ce200849eaad257fd"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.630147 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "f7e295e176ea4f86a3045772ca8c753c"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "76a88ee90a602faebe5e8c75f4900f03"
server_key_iv: "ad2f3c4c597fe85ce200849eaad257fd"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.633973 9058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 22:21:22.636415 9073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.637543 9058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:22.637636 9058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "f7e295e176ea4f86a3045772ca8c753c"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "76a88ee90a602faebe5e8c75f4900f03"
server_key_iv: "ad2f3c4c597fe85ce200849eaad257fd"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.637763 9058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:22.648437 9058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:22.651705 9058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:22.651917 9058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:22.653416 9058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:22.653465 9058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.653506 9058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:22.653558 9058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:22.664584 9058 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:38061
I20260619 22:21:22.664672 9188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:38061 every 8 connection(s)
I20260619 22:21:22.665663 9058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:22.667109 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.654281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:60503 (local address 127.1.74.190:42301)
0619 22:21:22.654407 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.654410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.655492 (+ 1082us) server_negotiation.cc:407] Connection header received
0619 22:21:22.656715 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.656718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.656778 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.656886 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.658229 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.658921 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.660194 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.660424 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.660545 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.663299 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.663303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.663306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.663340 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.664985 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.665443 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.665447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.665449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.665539 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.665881 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.665885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.665887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.666143 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.666240 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.666835 (+ 595us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.666982 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:21:22.667903 9082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.654821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.1.74.130:60503)
0619 22:21:22.655342 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.655383 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.656474 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.657056 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.657063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.657459 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:22.658025 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.658041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.659054 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.659058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.660057 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.660067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.660588 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.660593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.660795 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.661442 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:22.661484 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:22.663124 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:22.665090 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.665094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.665106 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.665340 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.665635 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.665637 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.665639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.665785 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.666294 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:22.666304 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:22.666612 (+ 308us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.666772 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.667085 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":135,"threads_started":1}
I20260619 22:21:22.668494 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9058
I20260619 22:21:22.668634 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:22.668942 1322 external_mini_cluster.cc:1428] Setting key 5c82a4c3204a05849474a65fdeba2529
I20260619 22:21:22.669299 9189 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42301
I20260619 22:21:22.669442 9189 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:22.669943 9189 heartbeater.cc:511] Master 127.1.74.190:42301 requested a full tablet report, sending...
I20260619 22:21:22.671010 8846 ts_manager.cc:194] Registered new tserver with Master: f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061)
I20260619 22:21:22.671758 8846 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:60503
WARNING: no policy specified for oryx/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:21:22.680269 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.671687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.1.74.130:38417)
0619 22:21:22.672010 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.672028 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.672119 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.672577 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.672581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.672883 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:22.673032 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.673042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.674183 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.674186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.674768 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.674776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.675510 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.675512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.675584 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.676218 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:22.676229 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:22.676575 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:22.678606 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.678608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.678612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.678805 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.679192 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.679194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.679195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.679274 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.679811 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:22.679815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:22.679917 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.680001 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.680117 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":95,"threads_started":1}
I20260619 22:21:22.680317 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.672133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:38417 (local address 127.1.74.190:42301)
0619 22:21:22.672258 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.672264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.672280 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:21:22.672322 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.672326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.672377 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.672459 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.673213 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.674060 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.674965 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.675388 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.675507 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.676742 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.676748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.676754 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.676789 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.678475 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.678991 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.678995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.678997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.679090 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.679369 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.679373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.679375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.679685 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.679801 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.680086 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.680201 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab.
Entry for principal oryx/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab.
I20260619 22:21:22.697336 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42301
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:22.806746 9198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:22.807137 9198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:22.807225 9198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:22.812145 9198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:22.812248 9198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:22.812316 9198 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 22:21:22.812382 9198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:22.818039 9198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.74.190:42301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:22.819185 9198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:22.820231 9198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:22.830217 9204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.830258 9203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:22.830219 9206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:22.830766 9198 server_base.cc:1034] running on GCE node
I20260619 22:21:22.831229 9198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:22.831930 9198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:22.833128 9198 hybrid_clock.cc:648] HybridClock initialized: now 1781907682833110 us; error 30 us; skew 500 ppm
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:22.836169 9198 init.cc:377] Logged in from keytab as oryx/127.1.74.131@KRBTEST.COM (short username oryx)
I20260619 22:21:22.837093 9198 webserver.cc:460] Webserver started at http://127.1.74.131:42899/ using document root <none> and password file <none>
I20260619 22:21:22.837715 9198 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:22.837811 9198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:22.838050 9198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:22.840209 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "9b843f1f213e4f60a4751637ec8f7fea"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "b88c2e08e638174b033b5d7a3d928172"
server_key_iv: "6a68bf60562ab3a2357ff2c05b0d2cab"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.840795 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "9b843f1f213e4f60a4751637ec8f7fea"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "b88c2e08e638174b033b5d7a3d928172"
server_key_iv: "6a68bf60562ab3a2357ff2c05b0d2cab"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.844928 9198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:22.847477 9213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.848763 9198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:22.848899 9198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "9b843f1f213e4f60a4751637ec8f7fea"
format_stamp: "Formatted at 2026-06-19 22:21:22 on dist-test-slave-k84h"
server_key: "b88c2e08e638174b033b5d7a3d928172"
server_key_iv: "6a68bf60562ab3a2357ff2c05b0d2cab"
server_key_version: "encryptionkey@0"
I20260619 22:21:22.849017 9198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:22.885823 9198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:22.889081 9198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:22.889310 9198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:22.891013 9198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:22.891090 9198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:22.891160 9198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:22.891209 9198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:22.902503 9198 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:37491
I20260619 22:21:22.902575 9328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:37491 every 8 connection(s)
I20260619 22:21:22.903491 9198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:22.904132 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9198
I20260619 22:21:22.904268 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:22.904583 1322 external_mini_cluster.cc:1428] Setting key 92a60422cc123d612911775017b8ab58
I20260619 22:21:22.905229 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.891791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:50445 (local address 127.1.74.190:42301)
0619 22:21:22.891950 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.891955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.892861 (+ 906us) server_negotiation.cc:407] Connection header received
0619 22:21:22.893923 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.893926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.893980 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.894032 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.895796 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.896447 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.897608 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.897833 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.897953 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.901037 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.901040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.901043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.901069 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.902835 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.903380 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.903384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.903386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.903477 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.903856 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.903860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.903862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.904113 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.904230 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.904875 (+ 645us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.905035 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 22:21:22.906222 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.892232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.1.74.131:50445)
0619 22:21:22.892695 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.892746 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.893666 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.894484 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.894491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.895042 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:22.895648 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.895674 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.896570 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.896573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.897475 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.897485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.898003 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.898007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.898217 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.898945 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:22.898971 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:22.900839 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:22.902978 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.902984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.903000 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.903272 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.903604 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.903607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.903619 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.903757 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.904289 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:22.904297 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:22.904611 (+ 314us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.904835 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.905164 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":136,"threads_started":1}
I20260619 22:21:22.907508 9329 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42301
I20260619 22:21:22.907693 9329 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:22.908203 9329 heartbeater.cc:511] Master 127.1.74.190:42301 requested a full tablet report, sending...
I20260619 22:21:22.909307 8845 ts_manager.cc:194] Registered new tserver with Master: 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:22.909991 8845 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:50445
I20260619 22:21:22.917258 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.909962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:34861 (local address 127.1.74.190:42301)
0619 22:21:22.910085 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.910091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.910107 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:21:22.910208 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.910210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.910277 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.910366 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.910886 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.911630 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.912559 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.912778 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.912860 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.914067 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.914070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.914072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.914095 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.915612 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.916009 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.916012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.916017 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.916129 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.916363 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.916365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.916366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.916565 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.916683 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.916951 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.917065 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:21:22.917289 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.909716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.1.74.131:34861)
0619 22:21:22.909999 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.910016 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.910108 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.910399 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.910402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.910632 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:22.910743 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.910749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.911804 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.911807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.912453 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.912462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.912900 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.912906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.912986 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.913564 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:22.913575 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:22.913922 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:22.915715 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.915717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.915719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.915905 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.916215 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:22.916217 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:22.916218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:22.916290 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:22.916673 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:22.916677 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:22.916781 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.916889 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.917086 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260619 22:21:22.918767 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 22:21:22.943513 8895 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:22.936974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.1.74.190:42301)
0619 22:21:22.937102 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.937105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.937118 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:21:22.937170 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.937173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.937212 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.937293 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.937994 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.938512 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.939414 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.939653 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.939745 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.943349 (+ 3604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58468: BlockingRecv error: recv got EOF from 127.0.0.1:58468 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 22:21:22 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:22.953392 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.945168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.1.74.190:42301)
0619 22:21:22.945321 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.945325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.945445 (+ 120us) server_negotiation.cc:407] Connection header received
0619 22:21:22.945604 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.945607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.945664 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.945750 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:22.946340 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.946924 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.947716 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.947945 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.948050 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.950314 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:22.950317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:22.950319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:22.950342 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:22.951599 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.952100 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.952103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.952105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.952189 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:22.952464 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:22.952468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:22.952470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:22.952658 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:22.952797 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.953089 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.953215 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 22:21:22.956723 8845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:22.959942 8845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:22.974303 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.968464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37491 (local address 127.0.0.1:54494)
0619 22:21:22.968840 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.968896 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.969097 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.969880 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.969886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.969908 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:22.970051 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.970058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.971373 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.971378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.972563 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.972573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.973654 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.973658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.973819 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.973876 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.973994 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.974129 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260619 22:21:22.974330 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.968616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54494 (local address 127.1.74.131:37491)
0619 22:21:22.969085 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.969098 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.969123 (+ 25us) server_negotiation.cc:407] Connection header received
0619 22:21:22.969285 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.969291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.969512 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.969708 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:22.970181 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.971265 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.972701 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.973542 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.973650 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.973807 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.974045 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.974156 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":131,"threads_started":1}
I20260619 22:21:22.974541 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.968716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:39365 (local address 127.0.0.1:32918)
0619 22:21:22.969362 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.969376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.969467 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.970408 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.970411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.970429 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:22.970525 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.970531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.971935 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.971938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.973094 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.973103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.974193 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.974196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.974290 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.974314 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.974365 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.974432 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260619 22:21:22.974994 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.969180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:38061 (local address 127.0.0.1:52214)
0619 22:21:22.969453 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:22.969467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:22.969550 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:22.970764 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:22.970767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:22.970784 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:22.970887 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.970893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.972138 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.972143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.973225 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:22.973235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.974360 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.974364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:22.974699 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.974724 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 22:21:22.974785 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 22:21:22.974877 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260619 22:21:22.975270 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.969359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32918 (local address 127.1.74.129:39365)
0619 22:21:22.969903 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.969913 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.969937 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:21:22.970015 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.970020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.970266 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.970410 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:22.970647 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.971814 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.973200 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.974068 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.974798 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.974980 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.975020 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.975148 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":78,"threads_started":1}
I20260619 22:21:22.975534 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:22.969795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52214 (local address 127.1.74.130:38061)
0619 22:21:22.970199 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:22.970210 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:22.970273 (+ 63us) server_negotiation.cc:407] Connection header received
0619 22:21:22.970417 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:22.970422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:22.970623 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:22.970769 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:22.971003 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.972022 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.973344 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:22.974161 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:22.974259 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:22.975272 (+ 1013us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:22.975324 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 22:21:22.975409 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":121,"threads_started":1}
I20260619 22:21:22.977502 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 9b45bb982f604d4f958a14dad24d9cd0 (DEFAULT_TABLE table=test-table [id=9a6603c0f1884b9081f04e0f858a795a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:22.977787 8983 tablet_service.cc:1467] Processing CreateTablet for tablet 9b45bb982f604d4f958a14dad24d9cd0 (DEFAULT_TABLE table=test-table [id=9a6603c0f1884b9081f04e0f858a795a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:22.978628 8983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b45bb982f604d4f958a14dad24d9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:22.978628 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b45bb982f604d4f958a14dad24d9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:22.984531 9123 tablet_service.cc:1467] Processing CreateTablet for tablet 9b45bb982f604d4f958a14dad24d9cd0 (DEFAULT_TABLE table=test-table [id=9a6603c0f1884b9081f04e0f858a795a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:22.985719 9123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b45bb982f604d4f958a14dad24d9cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:22.986127 9357 tablet_bootstrap.cc:492] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Bootstrap starting.
I20260619 22:21:22.987394 9358 tablet_bootstrap.cc:492] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Bootstrap starting.
I20260619 22:21:22.988437 9357 tablet_bootstrap.cc:654] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:22.989138 9358 tablet_bootstrap.cc:654] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:22.989391 9357 log.cc:824] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:22.989876 9358 log.cc:824] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:22.991513 9357 tablet_bootstrap.cc:492] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: No bootstrap required, opened a new log
I20260619 22:21:22.991706 9357 ts_tablet_manager.cc:1397] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:21:22.991739 9358 tablet_bootstrap.cc:492] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: No bootstrap required, opened a new log
I20260619 22:21:22.991942 9358 ts_tablet_manager.cc:1397] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 22:21:22.993706 9361 tablet_bootstrap.cc:492] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Bootstrap starting.
I20260619 22:21:22.995345 9358 raft_consensus.cc:348] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:22.995630 9358 raft_consensus.cc:374] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:22.995687 9358 raft_consensus.cc:729] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b843f1f213e4f60a4751637ec8f7fea, State: Initialized, Role: FOLLOWER
I20260619 22:21:22.996019 9361 tablet_bootstrap.cc:654] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:22.996018 9357 raft_consensus.cc:348] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:22.996147 9358 consensus_queue.cc:260] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:22.996307 9357 raft_consensus.cc:374] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:22.996359 9357 raft_consensus.cc:729] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Initialized, Role: FOLLOWER
I20260619 22:21:22.996831 9329 heartbeater.cc:503] Master 127.1.74.190:42301 was elected leader, sending a full tablet report...
I20260619 22:21:22.996798 9357 consensus_queue.cc:260] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:22.996863 9361 log.cc:824] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:22.997234 9358 ts_tablet_manager.cc:1428] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:21:22.997653 9357 ts_tablet_manager.cc:1428] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:21:22.997960 9049 heartbeater.cc:503] Master 127.1.74.190:42301 was elected leader, sending a full tablet report...
I20260619 22:21:22.999073 9361 tablet_bootstrap.cc:492] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: No bootstrap required, opened a new log
I20260619 22:21:22.999264 9361 ts_tablet_manager.cc:1397] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:21:23.003038 9361 raft_consensus.cc:348] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:23.003291 9361 raft_consensus.cc:374] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.003324 9361 raft_consensus.cc:729] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e295e176ea4f86a3045772ca8c753c, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.003777 9361 consensus_queue.cc:260] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:23.004465 9189 heartbeater.cc:503] Master 127.1.74.190:42301 was elected leader, sending a full tablet report...
I20260619 22:21:23.004786 9361 ts_tablet_manager.cc:1428] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260619 22:21:23.154979 9330 tablet.cc:2369] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:21:23.166980 9190 tablet.cc:2369] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:21:23.200244 9050 tablet.cc:2369] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:21:23.230829 9363 raft_consensus.cc:482] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:23.231223 9363 raft_consensus.cc:504] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:23.232465 9363 leader_election.cc:288] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491), f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061)
I20260619 22:21:23.238118 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.233355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:54127 (local address 127.1.74.130:38061)
0619 22:21:23.233487 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.233492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.233719 (+ 227us) server_negotiation.cc:407] Connection header received
0619 22:21:23.233780 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.233783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.233821 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.233880 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:23.234145 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.234879 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.236755 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.237367 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.237497 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.237553 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.237903 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.237977 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:23.238118 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.233020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:50243 (local address 127.1.74.131:37491)
0619 22:21:23.233205 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.233210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.233253 (+ 43us) server_negotiation.cc:407] Connection header received
0619 22:21:23.233462 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.233465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.233536 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.233605 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:23.233976 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.234785 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.236755 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.237367 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.237497 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.237569 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.237903 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.237977 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 22:21:23.238252 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.233246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:38061 (local address 127.1.74.129:54127)
0619 22:21:23.233601 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.233616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.233708 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.233915 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.233919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.233940 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:23.234041 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.234045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.234964 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.234966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.236600 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.236612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.237539 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.237544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.237711 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.237784 (+ 73us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.237884 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.238035 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":271,"thread_start_us":126,"threads_started":1}
I20260619 22:21:23.238252 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.232834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37491 (local address 127.1.74.129:50243)
0619 22:21:23.233141 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.233155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.233309 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.233652 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.233658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.233682 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:23.233838 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.233843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.234925 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.234930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.236600 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.236612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.237550 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.237554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.237719 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.237769 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.237881 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.238017 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":193,"spinlock_wait_cycles":3328,"thread_start_us":94,"threads_started":1}
I20260619 22:21:23.239089 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b45bb982f604d4f958a14dad24d9cd0" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c" is_pre_election: true
I20260619 22:21:23.239223 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b45bb982f604d4f958a14dad24d9cd0" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea" is_pre_election: true
I20260619 22:21:23.239475 9143 raft_consensus.cc:2454] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 0.
I20260619 22:21:23.239517 9283 raft_consensus.cc:2454] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 0.
I20260619 22:21:23.240033 8934 leader_election.cc:302] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b843f1f213e4f60a4751637ec8f7fea, ed51c42758a94de0b08bce0a6ac860e8; no voters:
I20260619 22:21:23.240342 9363 raft_consensus.cc:2781] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:23.240432 9363 raft_consensus.cc:482] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:23.240497 9363 raft_consensus.cc:3037] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.241750 9363 raft_consensus.cc:504] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:23.242230 9363 leader_election.cc:288] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 election: Requested vote from peers 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491), f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061)
I20260619 22:21:23.242735 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b45bb982f604d4f958a14dad24d9cd0" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c"
I20260619 22:21:23.242735 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b45bb982f604d4f958a14dad24d9cd0" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea"
I20260619 22:21:23.242908 9283 raft_consensus.cc:3037] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.242909 9143 raft_consensus.cc:3037] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.244354 9143 raft_consensus.cc:2454] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 1.
I20260619 22:21:23.244706 9283 raft_consensus.cc:2454] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 1.
I20260619 22:21:23.244726 8934 leader_election.cc:302] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed51c42758a94de0b08bce0a6ac860e8, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:23.244895 9363 raft_consensus.cc:2781] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:23.245158 9363 raft_consensus.cc:686] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 LEADER]: Becoming Leader. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Running, Role: LEADER
I20260619 22:21:23.245527 9363 consensus_queue.cc:237] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } }
I20260619 22:21:23.248690 8846 catalog_manager.cc:5495] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "ed51c42758a94de0b08bce0a6ac860e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } health_report { overall_health: HEALTHY } } }
I20260619 22:21:23.268513 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.264666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.1.74.129:39365)
0619 22:21:23.264814 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.264819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.264836 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:21:23.264968 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.264970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.265030 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.265166 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:23.265423 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.265972 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.266947 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.267198 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.267318 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.267615 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.268083 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.268232 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.268293 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.268356 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:23.275063 9283 raft_consensus.cc:1261] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Refusing update from remote peer ed51c42758a94de0b08bce0a6ac860e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.275115 9143 raft_consensus.cc:1261] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Refusing update from remote peer ed51c42758a94de0b08bce0a6ac860e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.275830 9368 consensus_queue.cc:1035] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.276065 9363 consensus_queue.cc:1035] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.294986 8846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:23.295192 8846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:23.297859 8846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Sending DeleteTablet for 3 replicas of tablet 9b45bb982f604d4f958a14dad24d9cd0
I20260619 22:21:23.298756 8983 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b45bb982f604d4f958a14dad24d9cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:32918
I20260619 22:21:23.298996 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b45bb982f604d4f958a14dad24d9cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:54494
I20260619 22:21:23.299365 9123 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b45bb982f604d4f958a14dad24d9cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:52214
I20260619 22:21:23.299512 9382 tablet_replica.cc:330] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: stopping tablet replica
I20260619 22:21:23.299724 8846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 22:21:23.299830 9383 tablet_replica.cc:330] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: stopping tablet replica
I20260619 22:21:23.299860 9382 raft_consensus.cc:2229] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:23.300127 9383 raft_consensus.cc:2229] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:23.300204 9382 raft_consensus.cc:2258] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 22:21:23.300485 8846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:23.300710 9381 tablet_replica.cc:330] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: stopping tablet replica
I20260619 22:21:23.301019 9381 raft_consensus.cc:2229] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:23.301864 9381 raft_consensus.cc:2258] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.302026 9382 ts_tablet_manager.cc:1905] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.302116 9383 raft_consensus.cc:2258] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.303936 9381 ts_tablet_manager.cc:1905] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.304919 9382 ts_tablet_manager.cc:1918] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:23.305025 9382 log.cc:1196] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/9b45bb982f604d4f958a14dad24d9cd0
I20260619 22:21:23.305349 9382 ts_tablet_manager.cc:1939] T 9b45bb982f604d4f958a14dad24d9cd0 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting consensus metadata
I20260619 22:21:23.306746 8983 tablet_service.cc:1467] Processing CreateTablet for tablet 9dddc6331b004f4bbc1a40812dc73ea8 (DEFAULT_TABLE table=test-table [id=b4b548f7ebb54bce8a7e4da0c28eaf85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.306798 9381 ts_tablet_manager.cc:1918] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:23.306921 9381 log.cc:1196] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/9b45bb982f604d4f958a14dad24d9cd0
I20260619 22:21:23.307053 8983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dddc6331b004f4bbc1a40812dc73ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.307341 9381 ts_tablet_manager.cc:1939] T 9b45bb982f604d4f958a14dad24d9cd0 P ed51c42758a94de0b08bce0a6ac860e8: Deleting consensus metadata
I20260619 22:21:23.308693 8830 catalog_manager.cc:4841] TS 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491): tablet 9b45bb982f604d4f958a14dad24d9cd0 (table test-table [id=9a6603c0f1884b9081f04e0f858a795a]) successfully deleted
I20260619 22:21:23.308944 9123 tablet_service.cc:1467] Processing CreateTablet for tablet 9dddc6331b004f4bbc1a40812dc73ea8 (DEFAULT_TABLE table=test-table [id=b4b548f7ebb54bce8a7e4da0c28eaf85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.309142 9357 tablet_bootstrap.cc:492] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Bootstrap starting.
I20260619 22:21:23.309201 9123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dddc6331b004f4bbc1a40812dc73ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.310092 9357 tablet_bootstrap.cc:654] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.310184 9383 ts_tablet_manager.cc:1905] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.310777 8831 catalog_manager.cc:4841] TS ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365): tablet 9b45bb982f604d4f958a14dad24d9cd0 (table test-table [id=9a6603c0f1884b9081f04e0f858a795a]) successfully deleted
I20260619 22:21:23.311661 9361 tablet_bootstrap.cc:492] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Bootstrap starting.
I20260619 22:21:23.311878 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 9dddc6331b004f4bbc1a40812dc73ea8 (DEFAULT_TABLE table=test-table [id=b4b548f7ebb54bce8a7e4da0c28eaf85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.312130 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dddc6331b004f4bbc1a40812dc73ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.312654 9361 tablet_bootstrap.cc:654] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.314271 9361 tablet_bootstrap.cc:492] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: No bootstrap required, opened a new log
I20260619 22:21:23.314350 9361 ts_tablet_manager.cc:1397] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 22:21:23.314378 9358 tablet_bootstrap.cc:492] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Bootstrap starting.
I20260619 22:21:23.314891 9361 raft_consensus.cc:348] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.315008 9361 raft_consensus.cc:374] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.315048 9361 raft_consensus.cc:729] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e295e176ea4f86a3045772ca8c753c, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.315344 9361 consensus_queue.cc:260] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.315407 9358 tablet_bootstrap.cc:654] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.315641 9357 tablet_bootstrap.cc:492] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: No bootstrap required, opened a new log
I20260619 22:21:23.315744 9357 ts_tablet_manager.cc:1397] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260619 22:21:23.316244 9357 raft_consensus.cc:348] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.316365 9357 raft_consensus.cc:374] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.316409 9357 raft_consensus.cc:729] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.316540 9357 consensus_queue.cc:260] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.316592 9361 ts_tablet_manager.cc:1428] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:23.316795 9357 ts_tablet_manager.cc:1428] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:23.316874 9383 ts_tablet_manager.cc:1918] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:23.316988 9383 log.cc:1196] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/9b45bb982f604d4f958a14dad24d9cd0
I20260619 22:21:23.317392 9383 ts_tablet_manager.cc:1939] T 9b45bb982f604d4f958a14dad24d9cd0 P f7e295e176ea4f86a3045772ca8c753c: Deleting consensus metadata
I20260619 22:21:23.318367 9358 tablet_bootstrap.cc:492] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: No bootstrap required, opened a new log
I20260619 22:21:23.318490 9358 ts_tablet_manager.cc:1397] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260619 22:21:23.318966 9358 raft_consensus.cc:348] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.319082 9358 raft_consensus.cc:374] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.319126 9358 raft_consensus.cc:729] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b843f1f213e4f60a4751637ec8f7fea, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.319252 9358 consensus_queue.cc:260] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.319502 9358 ts_tablet_manager.cc:1428] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 22:21:23.319744 8830 catalog_manager.cc:4841] TS f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061): tablet 9b45bb982f604d4f958a14dad24d9cd0 (table test-table [id=9a6603c0f1884b9081f04e0f858a795a]) successfully deleted
I20260619 22:21:23.486829 9363 raft_consensus.cc:482] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:23.487022 9363 raft_consensus.cc:504] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.487522 9363 leader_election.cc:288] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:23.488117 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddc6331b004f4bbc1a40812dc73ea8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c" is_pre_election: true
I20260619 22:21:23.488117 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddc6331b004f4bbc1a40812dc73ea8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea" is_pre_election: true
I20260619 22:21:23.488279 9143 raft_consensus.cc:2454] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 0.
I20260619 22:21:23.488279 9283 raft_consensus.cc:2454] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 0.
I20260619 22:21:23.488626 8934 leader_election.cc:302] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed51c42758a94de0b08bce0a6ac860e8, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:23.488853 9363 raft_consensus.cc:2781] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:23.488919 9363 raft_consensus.cc:482] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:23.488973 9363 raft_consensus.cc:3037] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.489984 9363 raft_consensus.cc:504] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.490362 9363 leader_election.cc:288] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 election: Requested vote from peers f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:23.490806 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddc6331b004f4bbc1a40812dc73ea8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c"
I20260619 22:21:23.490806 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddc6331b004f4bbc1a40812dc73ea8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea"
I20260619 22:21:23.490962 9283 raft_consensus.cc:3037] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.490962 9143 raft_consensus.cc:3037] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.492137 9143 raft_consensus.cc:2454] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 1.
I20260619 22:21:23.492137 9283 raft_consensus.cc:2454] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 1.
I20260619 22:21:23.492431 8934 leader_election.cc:302] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed51c42758a94de0b08bce0a6ac860e8, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:23.492646 9363 raft_consensus.cc:2781] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:23.492753 9363 raft_consensus.cc:686] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 LEADER]: Becoming Leader. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Running, Role: LEADER
I20260619 22:21:23.492895 9363 consensus_queue.cc:237] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.494663 8846 catalog_manager.cc:5495] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "ed51c42758a94de0b08bce0a6ac860e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } health_report { overall_health: UNKNOWN } } }
Jun 19 22:21:23 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907682, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.190@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:23.541458 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.532661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42301 (local address 127.0.0.1:58490)
0619 22:21:23.532978 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.532992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.533087 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.533299 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.533301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.533674 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:23.533778 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.533784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.534565 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.534568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.535214 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.535224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.535643 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.535645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.535703 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.536289 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:23.536301 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:23.537844 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:23.539825 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:23.539834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:23.539839 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:23.540064 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:23.540336 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:23.540338 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:23.540339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:23.540412 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:23.540899 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:23.540905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:23.541009 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.541142 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.541320 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":102,"threads_started":1}
I20260619 22:21:23.541544 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.532745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.1.74.190:42301)
0619 22:21:23.533084 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.533090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.533105 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:23.533147 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.533149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.533187 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.533268 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:23.533905 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.534462 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.535318 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.535539 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.535626 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.537984 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:23.537987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:23.537989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:23.538011 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:23.539718 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:23.540154 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:23.540157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:23.540158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:23.540254 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:23.540520 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:23.540523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:23.540524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:23.540791 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:23.540892 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.541274 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.541385 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":141,"threads_started":1}
I20260619 22:21:23.545718 8846 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58490:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 22:21:23.546404 8846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:23.551313 8983 tablet_service.cc:1467] Processing CreateTablet for tablet 21d7620cc62e4533ab418bf1e01231c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5a885515914457bc7760282fef5a99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:21:23.551411 9123 tablet_service.cc:1467] Processing CreateTablet for tablet 21d7620cc62e4533ab418bf1e01231c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5a885515914457bc7760282fef5a99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:21:23.551420 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 21d7620cc62e4533ab418bf1e01231c6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5a885515914457bc7760282fef5a99]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 22:21:23.551772 8983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d7620cc62e4533ab418bf1e01231c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.551798 9123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d7620cc62e4533ab418bf1e01231c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.551793 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d7620cc62e4533ab418bf1e01231c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.554560 9357 tablet_bootstrap.cc:492] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: Bootstrap starting.
I20260619 22:21:23.554710 9361 tablet_bootstrap.cc:492] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: Bootstrap starting.
I20260619 22:21:23.555519 9357 tablet_bootstrap.cc:654] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.555713 9361 tablet_bootstrap.cc:654] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.556537 9358 tablet_bootstrap.cc:492] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: Bootstrap starting.
I20260619 22:21:23.556684 9357 tablet_bootstrap.cc:492] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: No bootstrap required, opened a new log
I20260619 22:21:23.556761 9357 ts_tablet_manager.cc:1397] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:23.557008 9361 tablet_bootstrap.cc:492] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: No bootstrap required, opened a new log
I20260619 22:21:23.557086 9361 ts_tablet_manager.cc:1397] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:23.557158 9357 raft_consensus.cc:348] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.557288 9357 raft_consensus.cc:374] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.557325 9357 raft_consensus.cc:729] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.557449 9357 consensus_queue.cc:260] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.557660 9358 tablet_bootstrap.cc:654] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.557720 9357 ts_tablet_manager.cc:1428] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:23.557497 9361 raft_consensus.cc:348] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.557786 9361 raft_consensus.cc:374] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.557822 9361 raft_consensus.cc:729] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e295e176ea4f86a3045772ca8c753c, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.557961 9361 consensus_queue.cc:260] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.557964 9363 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.558086 9363 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.558240 9361 ts_tablet_manager.cc:1428] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:23.558543 9365 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.558681 9365 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.559273 9358 tablet_bootstrap.cc:492] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: No bootstrap required, opened a new log
I20260619 22:21:23.559392 9358 ts_tablet_manager.cc:1397] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:23.559828 9358 raft_consensus.cc:348] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.559948 9358 raft_consensus.cc:374] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.559984 9358 raft_consensus.cc:729] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b843f1f213e4f60a4751637ec8f7fea, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.560118 9358 consensus_queue.cc:260] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.560398 9362 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.560523 9362 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.561264 9358 ts_tablet_manager.cc:1428] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:23.592739 9362 raft_consensus.cc:482] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:23.592895 9362 raft_consensus.cc:504] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.594208 9362 leader_election.cc:288] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365), f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061)
I20260619 22:21:23.598311 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.594449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:38061 (local address 127.1.74.131:37899)
0619 22:21:23.594989 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.595001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.595114 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.595337 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.595340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.595354 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:23.595500 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.595505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.596256 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.596262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.597180 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.597189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.597822 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.597828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.597941 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.597969 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.598048 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.598104 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":58,"spinlock_wait_cycles":23936,"thread_start_us":85,"threads_started":1}
I20260619 22:21:23.598313 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.594752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:37899 (local address 127.1.74.130:38061)
0619 22:21:23.594958 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.594965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.595100 (+ 135us) server_negotiation.cc:407] Connection header received
0619 22:21:23.595199 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.595201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.595240 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.595297 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:23.595617 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.596137 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.597309 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.597698 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.597795 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.597875 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.598094 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.598183 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260619 22:21:23.598399 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.594523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:34473 (local address 127.1.74.129:39365)
0619 22:21:23.594653 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.594660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.594905 (+ 245us) server_negotiation.cc:407] Connection header received
0619 22:21:23.595184 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.595188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.595251 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.595342 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:23.595669 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.596165 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.597344 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.597767 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.597886 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.597950 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.598197 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.598273 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:21:23.598366 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.594407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:39365 (local address 127.1.74.131:34473)
0619 22:21:23.594776 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.594788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.595010 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.595368 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.595370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.595399 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:23.595520 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.595525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.596302 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.596305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.597187 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.597196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.597895 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.597900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.598053 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.598079 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.598156 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.598218 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":89,"thread_start_us":111,"threads_started":1}
I20260619 22:21:23.599110 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7620cc62e4533ab418bf1e01231c6" candidate_uuid: "9b843f1f213e4f60a4751637ec8f7fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c" is_pre_election: true
I20260619 22:21:23.599161 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7620cc62e4533ab418bf1e01231c6" candidate_uuid: "9b843f1f213e4f60a4751637ec8f7fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed51c42758a94de0b08bce0a6ac860e8" is_pre_election: true
I20260619 22:21:23.599282 9143 raft_consensus.cc:2454] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b843f1f213e4f60a4751637ec8f7fea in term 0.
I20260619 22:21:23.599460 9003 raft_consensus.cc:2454] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b843f1f213e4f60a4751637ec8f7fea in term 0.
I20260619 22:21:23.599656 9214 leader_election.cc:302] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b843f1f213e4f60a4751637ec8f7fea, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:23.599951 9362 raft_consensus.cc:2781] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:23.600028 9362 raft_consensus.cc:482] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:23.600064 9362 raft_consensus.cc:3037] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.601033 9362 raft_consensus.cc:504] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.601392 9362 leader_election.cc:288] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [CANDIDATE]: Term 1 election: Requested vote from peers ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365), f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061)
I20260619 22:21:23.601725 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7620cc62e4533ab418bf1e01231c6" candidate_uuid: "9b843f1f213e4f60a4751637ec8f7fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed51c42758a94de0b08bce0a6ac860e8"
I20260619 22:21:23.601848 9003 raft_consensus.cc:3037] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.601840 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7620cc62e4533ab418bf1e01231c6" candidate_uuid: "9b843f1f213e4f60a4751637ec8f7fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c"
I20260619 22:21:23.601969 9143 raft_consensus.cc:3037] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.602932 9003 raft_consensus.cc:2454] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b843f1f213e4f60a4751637ec8f7fea in term 1.
I20260619 22:21:23.602944 9143 raft_consensus.cc:2454] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b843f1f213e4f60a4751637ec8f7fea in term 1.
I20260619 22:21:23.603226 9215 leader_election.cc:302] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b843f1f213e4f60a4751637ec8f7fea, ed51c42758a94de0b08bce0a6ac860e8; no voters:
I20260619 22:21:23.603442 9362 raft_consensus.cc:2781] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:23.603624 9362 raft_consensus.cc:686] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 LEADER]: Becoming Leader. State: Replica: 9b843f1f213e4f60a4751637ec8f7fea, State: Running, Role: LEADER
I20260619 22:21:23.603888 9362 consensus_queue.cc:237] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.605266 9400 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: TxnStatusTablet state changed. Reason: New leader 9b843f1f213e4f60a4751637ec8f7fea. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.605382 9400 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:21:23.605645 9402 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 22:21:23.605809 9402 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 22:21:23.606415 8845 catalog_manager.cc:5495] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131). New cstate: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } health_report { overall_health: HEALTHY } } }
I20260619 22:21:23.627372 9393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.623362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37491 (local address 127.0.0.1:54506)
0619 22:21:23.623644 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.623661 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.623804 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.624148 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.624151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.624160 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:21:23.624270 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.624276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.625007 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.625013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.625651 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.625659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.626149 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.626152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.626244 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.626298 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.627039 (+ 741us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.627056 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.627132 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.627224 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260619 22:21:23.627372 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.623483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.1.74.131:37491)
0619 22:21:23.623627 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.623632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.623737 (+ 105us) server_negotiation.cc:407] Connection header received
0619 22:21:23.623970 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.623973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.624026 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.624135 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:23.624387 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.624895 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.625777 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.625995 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.626137 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.626449 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.626945 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.627067 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.627145 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.627200 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:21:23.630256 9143 raft_consensus.cc:1261] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Refusing update from remote peer 9b843f1f213e4f60a4751637ec8f7fea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.630509 9003 raft_consensus.cc:1261] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Refusing update from remote peer 9b843f1f213e4f60a4751637ec8f7fea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.630795 9362 consensus_queue.cc:1035] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.631081 9400 consensus_queue.cc:1035] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.632838 9363 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: TxnStatusTablet state changed. Reason: New leader 9b843f1f213e4f60a4751637ec8f7fea. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.632851 9365 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: TxnStatusTablet state changed. Reason: New leader 9b843f1f213e4f60a4751637ec8f7fea. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.632958 9365 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.632958 9363 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.634968 9400 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.635110 9400 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:21:23.635215 9373 mvcc.cc:205] Tried to move back new op lower bound from 7298693872144297984 to 7298693872044326912. Current Snapshot: MvccSnapshot[applied={T|T < 7298693872144297984}]
I20260619 22:21:23.635301 9372 mvcc.cc:205] Tried to move back new op lower bound from 7298693872144297984 to 7298693872044326912. Current Snapshot: MvccSnapshot[applied={T|T < 7298693872144297984}]
I20260619 22:21:23.635519 9365 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.635613 9365 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P f7e295e176ea4f86a3045772ca8c753c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.637939 9362 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.638059 9362 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P 9b843f1f213e4f60a4751637ec8f7fea: This TxnStatusTablet replica's current role is: LEADER
I20260619 22:21:23.639526 9363 tablet_replica.cc:439] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b843f1f213e4f60a4751637ec8f7fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } } }
I20260619 22:21:23.639658 9363 tablet_replica.cc:442] T 21d7620cc62e4533ab418bf1e01231c6 P ed51c42758a94de0b08bce0a6ac860e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 22:21:23.639812 9371 mvcc.cc:205] Tried to move back new op lower bound from 7298693872144297984 to 7298693872044326912. Current Snapshot: MvccSnapshot[applied={T|T < 7298693872144297984 or (T in {7298693872144297984})}]
I20260619 22:21:23.650226 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.646521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37491 (local address 127.1.74.129:34381)
0619 22:21:23.646830 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.646889 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.646997 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.647260 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.647262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.647269 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:21:23.647362 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.647366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.648348 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.648350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.648951 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.648959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.649522 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.649524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.649617 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.649662 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.649963 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.649969 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.650026 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.650098 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":124,"threads_started":1}
I20260619 22:21:23.650228 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.646639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:34381 (local address 127.1.74.131:37491)
0619 22:21:23.646771 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.646776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.646950 (+ 174us) server_negotiation.cc:407] Connection header received
0619 22:21:23.647084 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.647090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.647162 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.647246 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:23.647465 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.648247 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.649071 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.649410 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.649517 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.649792 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.649876 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.649948 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.650040 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.650089 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:21:23.657703 9143 raft_consensus.cc:1261] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Refusing update from remote peer ed51c42758a94de0b08bce0a6ac860e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.658288 9283 raft_consensus.cc:1261] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Refusing update from remote peer ed51c42758a94de0b08bce0a6ac860e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.658661 9363 consensus_queue.cc:1035] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.658880 9368 consensus_queue.cc:1035] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.686594 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.680554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:48447 (local address 127.1.74.129:39365)
0619 22:21:23.680688 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.680693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.683200 (+ 2507us) server_negotiation.cc:407] Connection header received
0619 22:21:23.683420 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.683423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.683483 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.683541 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:23.683891 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.684405 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.685304 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.685630 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.685751 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.686058 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.686156 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.686206 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.686382 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.686446 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:23.687273 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.680503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:39365 (local address 127.1.74.131:48447)
0619 22:21:23.683111 (+ 2608us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.683131 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.683286 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.683667 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.683669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.683676 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:21:23.683783 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.683786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.684516 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.684522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.685181 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.685190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.685751 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.685754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.685848 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.685903 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.686273 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.686281 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.686987 (+ 706us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.687085 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2458,"thread_start_us":124,"threads_started":1}
I20260619 22:21:23.712249 8845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:23.712463 8845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:23.714555 8845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Sending DeleteTablet for 3 replicas of tablet 9dddc6331b004f4bbc1a40812dc73ea8
I20260619 22:21:23.715323 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dddc6331b004f4bbc1a40812dc73ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:54494
I20260619 22:21:23.715323 9123 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dddc6331b004f4bbc1a40812dc73ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:52214
I20260619 22:21:23.715516 9383 tablet_replica.cc:330] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: stopping tablet replica
I20260619 22:21:23.715523 9382 tablet_replica.cc:330] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: stopping tablet replica
I20260619 22:21:23.715638 9383 raft_consensus.cc:2229] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:23.715667 9382 raft_consensus.cc:2229] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:23.715754 9383 raft_consensus.cc:2258] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.715801 9382 raft_consensus.cc:2258] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.718567 8983 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dddc6331b004f4bbc1a40812dc73ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:32918
I20260619 22:21:23.718778 9381 tablet_replica.cc:330] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: stopping tablet replica
I20260619 22:21:23.718914 9382 ts_tablet_manager.cc:1905] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.718940 9381 raft_consensus.cc:2229] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:23.719162 9381 raft_consensus.cc:2258] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.719903 9383 ts_tablet_manager.cc:1905] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.721745 9382 ts_tablet_manager.cc:1918] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:21:23.721812 9382 log.cc:1196] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/9dddc6331b004f4bbc1a40812dc73ea8
I20260619 22:21:23.722069 9382 ts_tablet_manager.cc:1939] T 9dddc6331b004f4bbc1a40812dc73ea8 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting consensus metadata
I20260619 22:21:23.722859 8830 catalog_manager.cc:4841] TS 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491): tablet 9dddc6331b004f4bbc1a40812dc73ea8 (table test-table [id=b4b548f7ebb54bce8a7e4da0c28eaf85]) successfully deleted
I20260619 22:21:23.723155 9383 ts_tablet_manager.cc:1918] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:21:23.723237 9383 log.cc:1196] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/9dddc6331b004f4bbc1a40812dc73ea8
I20260619 22:21:23.723512 9383 ts_tablet_manager.cc:1939] T 9dddc6331b004f4bbc1a40812dc73ea8 P f7e295e176ea4f86a3045772ca8c753c: Deleting consensus metadata
I20260619 22:21:23.723551 9381 ts_tablet_manager.cc:1905] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.724239 8830 catalog_manager.cc:4841] TS f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061): tablet 9dddc6331b004f4bbc1a40812dc73ea8 (table test-table [id=b4b548f7ebb54bce8a7e4da0c28eaf85]) successfully deleted
I20260619 22:21:23.726126 9381 ts_tablet_manager.cc:1918] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:21:23.726202 9381 log.cc:1196] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/9dddc6331b004f4bbc1a40812dc73ea8
I20260619 22:21:23.726436 9381 ts_tablet_manager.cc:1939] T 9dddc6331b004f4bbc1a40812dc73ea8 P ed51c42758a94de0b08bce0a6ac860e8: Deleting consensus metadata
I20260619 22:21:23.727133 8831 catalog_manager.cc:4841] TS ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365): tablet 9dddc6331b004f4bbc1a40812dc73ea8 (table test-table [id=b4b548f7ebb54bce8a7e4da0c28eaf85]) successfully deleted
Jun 19 22:21:23 dist-test-slave-k84h krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 22:21:23 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:23 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 22:21:23.746167 9394 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:23.739189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58502 (local address 127.1.74.190:42301)
0619 22:21:23.739325 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.739331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.739526 (+ 195us) server_negotiation.cc:407] Connection header received
0619 22:21:23.739730 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.739734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.739789 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.739877 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:23.740629 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.741380 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.742390 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.742636 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.742751 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.746022 (+ 3271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58502: BlockingRecv error: recv got EOF from 127.0.0.1:58502 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 22:21:23 dist-test-slave-k84h krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM
I20260619 22:21:23.757155 9394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.747977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58508 (local address 127.1.74.190:42301)
0619 22:21:23.748103 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.748106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.748267 (+ 161us) server_negotiation.cc:407] Connection header received
0619 22:21:23.748391 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.748393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.748430 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.748495 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:23.749233 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.749747 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.750801 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.751088 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.751174 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.753999 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:23.754003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:23.754007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:23.754035 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:23.755470 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:23.755913 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:23.755921 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:23.755923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:23.755992 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:23.756280 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:23.756283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:23.756285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:23.756480 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:23.756614 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.756855 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.756974 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 22:21:23.760095 8845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:23.760599 8845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:23.765538 8983 tablet_service.cc:1467] Processing CreateTablet for tablet 1ada9efabf0f427480da398e35c6b2e8 (DEFAULT_TABLE table=test-table [id=5eff3d3ba74d4659916c71a60e174383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.765590 9123 tablet_service.cc:1467] Processing CreateTablet for tablet 1ada9efabf0f427480da398e35c6b2e8 (DEFAULT_TABLE table=test-table [id=5eff3d3ba74d4659916c71a60e174383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.765899 8983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ada9efabf0f427480da398e35c6b2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.765904 9123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ada9efabf0f427480da398e35c6b2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.767856 9357 tablet_bootstrap.cc:492] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Bootstrap starting.
I20260619 22:21:23.768246 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 1ada9efabf0f427480da398e35c6b2e8 (DEFAULT_TABLE table=test-table [id=5eff3d3ba74d4659916c71a60e174383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.768404 9361 tablet_bootstrap.cc:492] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Bootstrap starting.
I20260619 22:21:23.768491 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ada9efabf0f427480da398e35c6b2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.768712 9357 tablet_bootstrap.cc:654] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.769273 9361 tablet_bootstrap.cc:654] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.770591 9361 tablet_bootstrap.cc:492] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: No bootstrap required, opened a new log
I20260619 22:21:23.770682 9361 ts_tablet_manager.cc:1397] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:23.770984 9358 tablet_bootstrap.cc:492] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Bootstrap starting.
I20260619 22:21:23.771302 9361 raft_consensus.cc:348] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.771431 9361 raft_consensus.cc:374] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.771476 9361 raft_consensus.cc:729] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e295e176ea4f86a3045772ca8c753c, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.771606 9361 consensus_queue.cc:260] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.771878 9358 tablet_bootstrap.cc:654] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.771879 9361 ts_tablet_manager.cc:1428] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:23.773102 9358 tablet_bootstrap.cc:492] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: No bootstrap required, opened a new log
I20260619 22:21:23.773200 9358 ts_tablet_manager.cc:1397] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:23.774072 9358 raft_consensus.cc:348] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.774192 9358 raft_consensus.cc:374] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.774232 9358 raft_consensus.cc:729] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b843f1f213e4f60a4751637ec8f7fea, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.774360 9358 consensus_queue.cc:260] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.774654 9358 ts_tablet_manager.cc:1428] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:23.775712 9357 tablet_bootstrap.cc:492] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: No bootstrap required, opened a new log
I20260619 22:21:23.775823 9357 ts_tablet_manager.cc:1397] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 22:21:23.776268 9357 raft_consensus.cc:348] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.776402 9357 raft_consensus.cc:374] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.776445 9357 raft_consensus.cc:729] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.776577 9357 consensus_queue.cc:260] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.776834 9357 ts_tablet_manager.cc:1428] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:21:23.893095 9363 raft_consensus.cc:482] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:23.893293 9363 raft_consensus.cc:504] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.893591 9365 raft_consensus.cc:482] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:23.893703 9365 raft_consensus.cc:504] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.893839 9363 leader_election.cc:288] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:23.894472 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ada9efabf0f427480da398e35c6b2e8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c" is_pre_election: true
I20260619 22:21:23.894539 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ada9efabf0f427480da398e35c6b2e8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea" is_pre_election: true
I20260619 22:21:23.894613 9143 raft_consensus.cc:2454] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 0.
I20260619 22:21:23.894696 9283 raft_consensus.cc:2454] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 0.
I20260619 22:21:23.894984 8934 leader_election.cc:302] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed51c42758a94de0b08bce0a6ac860e8, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:23.895073 9365 leader_election.cc:288] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:23.895323 9363 raft_consensus.cc:2781] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:23.895383 9363 raft_consensus.cc:482] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:23.895440 9363 raft_consensus.cc:3037] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.896744 9363 raft_consensus.cc:504] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.897145 9363 leader_election.cc:288] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 election: Requested vote from peers f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:23.897552 9143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ada9efabf0f427480da398e35c6b2e8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e295e176ea4f86a3045772ca8c753c"
I20260619 22:21:23.897678 9143 raft_consensus.cc:3037] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.897665 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ada9efabf0f427480da398e35c6b2e8" candidate_uuid: "ed51c42758a94de0b08bce0a6ac860e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea"
I20260619 22:21:23.897792 9283 raft_consensus.cc:3037] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:23.898814 9143 raft_consensus.cc:2454] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 1.
I20260619 22:21:23.898975 9283 raft_consensus.cc:2454] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed51c42758a94de0b08bce0a6ac860e8 in term 1.
I20260619 22:21:23.899191 8934 leader_election.cc:302] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed51c42758a94de0b08bce0a6ac860e8, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:23.899521 9363 raft_consensus.cc:2781] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:23.899921 9363 raft_consensus.cc:686] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 LEADER]: Becoming Leader. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Running, Role: LEADER
I20260619 22:21:23.900085 9363 consensus_queue.cc:237] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.900511 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.895643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:51265 (local address 127.1.74.131:37491)
0619 22:21:23.895780 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.895784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.895800 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:21:23.895965 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.895969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.896026 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.896074 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:23.896509 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.897327 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.899182 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.899763 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.899871 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.899923 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.900302 (+ 379us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.900387 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:21:23.900638 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.895886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:43957 (local address 127.1.74.129:39365)
0619 22:21:23.895996 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.896000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.896012 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:23.896065 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.896069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.896113 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.896187 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:23.896621 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.897326 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.899233 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.899742 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.899857 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.899909 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.900403 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.900491 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 22:21:23.900897 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.895324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37491 (local address 127.1.74.130:51265)
0619 22:21:23.895752 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.895764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.895823 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.896225 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.896228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.896245 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:23.896389 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.896394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.897481 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.897485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.898988 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.899002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.900004 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.900008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.900150 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.900175 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.900630 (+ 455us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.900710 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":99,"spinlock_wait_cycles":2944,"thread_start_us":78,"threads_started":1}
I20260619 22:21:23.901279 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.895281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:39365 (local address 127.1.74.130:43957)
0619 22:21:23.895679 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:23.895695 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:23.895809 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:23.896239 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:23.896242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:23.896291 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:23.896451 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.896458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.897467 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.897471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.898983 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:23.898995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.899987 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.899991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:23.900261 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.900284 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 22:21:23.900873 (+ 589us) client_negotiation.cc:240] Negotiation successful
0619 22:21:23.901139 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":177,"spinlock_wait_cycles":33024,"thread_start_us":110,"threads_started":1}
I20260619 22:21:23.901362 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ada9efabf0f427480da398e35c6b2e8" candidate_uuid: "f7e295e176ea4f86a3045772ca8c753c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea" is_pre_election: true
I20260619 22:21:23.901540 9283 raft_consensus.cc:2379] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7e295e176ea4f86a3045772ca8c753c in current term 1: Already voted for candidate ed51c42758a94de0b08bce0a6ac860e8 in this term.
I20260619 22:21:23.902021 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ada9efabf0f427480da398e35c6b2e8" candidate_uuid: "f7e295e176ea4f86a3045772ca8c753c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed51c42758a94de0b08bce0a6ac860e8" is_pre_election: true
I20260619 22:21:23.902417 9075 leader_election.cc:302] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7e295e176ea4f86a3045772ca8c753c; no voters: 9b843f1f213e4f60a4751637ec8f7fea, ed51c42758a94de0b08bce0a6ac860e8
I20260619 22:21:23.902479 8845 catalog_manager.cc:5495] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "ed51c42758a94de0b08bce0a6ac860e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } health_report { overall_health: UNKNOWN } } }
I20260619 22:21:23.902668 9365 raft_consensus.cc:2726] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 22:21:23.931840 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:23.928604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.1.74.129:39365)
0619 22:21:23.928770 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:23.928775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:23.928790 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:23.928909 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:23.928912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:23.928955 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:23.929036 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:23.929292 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.929798 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.930648 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:23.930949 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:23.931074 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:23.931385 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:23.931463 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:23.931548 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:23.931643 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 22:21:23.931693 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 22:21:23.933801 9283 raft_consensus.cc:1261] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Refusing update from remote peer ed51c42758a94de0b08bce0a6ac860e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.933872 9143 raft_consensus.cc:1261] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Refusing update from remote peer ed51c42758a94de0b08bce0a6ac860e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:23.934271 9363 consensus_queue.cc:1035] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.934435 9368 consensus_queue.cc:1035] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:23.941715 8845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:23.941890 8845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:23.943979 8845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Sending DeleteTablet for 3 replicas of tablet 1ada9efabf0f427480da398e35c6b2e8
I20260619 22:21:23.944695 8983 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ada9efabf0f427480da398e35c6b2e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:32918
I20260619 22:21:23.944706 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ada9efabf0f427480da398e35c6b2e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:54494
I20260619 22:21:23.944883 9381 tablet_replica.cc:330] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: stopping tablet replica
I20260619 22:21:23.944883 9382 tablet_replica.cc:330] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: stopping tablet replica
I20260619 22:21:23.945000 9381 raft_consensus.cc:2229] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:23.944998 9382 raft_consensus.cc:2229] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:23.945066 9123 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ada9efabf0f427480da398e35c6b2e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:23 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:52214
I20260619 22:21:23.945133 9382 raft_consensus.cc:2258] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.945222 9381 raft_consensus.cc:2258] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.945308 9383 tablet_replica.cc:330] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: stopping tablet replica
I20260619 22:21:23.945441 9383 raft_consensus.cc:2229] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:23.945556 9383 raft_consensus.cc:2258] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:23.945914 9381 ts_tablet_manager.cc:1905] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.946201 9383 ts_tablet_manager.cc:1905] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.946156 8845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:23.946506 8845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:23.948442 9383 ts_tablet_manager.cc:1918] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:23.948442 9381 ts_tablet_manager.cc:1918] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:23.948511 9381 log.cc:1196] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/1ada9efabf0f427480da398e35c6b2e8
I20260619 22:21:23.948511 9383 log.cc:1196] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/1ada9efabf0f427480da398e35c6b2e8
I20260619 22:21:23.948774 9383 ts_tablet_manager.cc:1939] T 1ada9efabf0f427480da398e35c6b2e8 P f7e295e176ea4f86a3045772ca8c753c: Deleting consensus metadata
I20260619 22:21:23.948774 9381 ts_tablet_manager.cc:1939] T 1ada9efabf0f427480da398e35c6b2e8 P ed51c42758a94de0b08bce0a6ac860e8: Deleting consensus metadata
I20260619 22:21:23.949407 9382 ts_tablet_manager.cc:1905] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:23.949582 8830 catalog_manager.cc:4841] TS f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061): tablet 1ada9efabf0f427480da398e35c6b2e8 (table test-table [id=5eff3d3ba74d4659916c71a60e174383]) successfully deleted
I20260619 22:21:23.951753 9382 ts_tablet_manager.cc:1918] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:23.951838 9382 log.cc:1196] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/1ada9efabf0f427480da398e35c6b2e8
I20260619 22:21:23.952124 9382 ts_tablet_manager.cc:1939] T 1ada9efabf0f427480da398e35c6b2e8 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting consensus metadata
I20260619 22:21:23.952505 9123 tablet_service.cc:1467] Processing CreateTablet for tablet 496850bee9f348d484b2a797751c72a6 (DEFAULT_TABLE table=test-table [id=f5be5164046a4b7682c0ee964e92b0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.952598 9263 tablet_service.cc:1467] Processing CreateTablet for tablet 496850bee9f348d484b2a797751c72a6 (DEFAULT_TABLE table=test-table [id=f5be5164046a4b7682c0ee964e92b0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.952780 9123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 496850bee9f348d484b2a797751c72a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.952809 9263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 496850bee9f348d484b2a797751c72a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.952829 8830 catalog_manager.cc:4841] TS 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491): tablet 1ada9efabf0f427480da398e35c6b2e8 (table test-table [id=5eff3d3ba74d4659916c71a60e174383]) successfully deleted
I20260619 22:21:23.953404 8983 tablet_service.cc:1467] Processing CreateTablet for tablet 496850bee9f348d484b2a797751c72a6 (DEFAULT_TABLE table=test-table [id=f5be5164046a4b7682c0ee964e92b0dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:23.953650 8983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 496850bee9f348d484b2a797751c72a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:23.953776 8831 catalog_manager.cc:4841] TS ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365): tablet 1ada9efabf0f427480da398e35c6b2e8 (table test-table [id=5eff3d3ba74d4659916c71a60e174383]) successfully deleted
I20260619 22:21:23.954885 9358 tablet_bootstrap.cc:492] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Bootstrap starting.
I20260619 22:21:23.954975 9361 tablet_bootstrap.cc:492] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Bootstrap starting.
I20260619 22:21:23.955595 9357 tablet_bootstrap.cc:492] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: Bootstrap starting.
I20260619 22:21:23.955739 9358 tablet_bootstrap.cc:654] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.955767 9361 tablet_bootstrap.cc:654] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.956437 9357 tablet_bootstrap.cc:654] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:23.956936 9361 tablet_bootstrap.cc:492] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: No bootstrap required, opened a new log
I20260619 22:21:23.957015 9361 ts_tablet_manager.cc:1397] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:23.957038 9358 tablet_bootstrap.cc:492] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: No bootstrap required, opened a new log
I20260619 22:21:23.957099 9358 ts_tablet_manager.cc:1397] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:23.957414 9357 tablet_bootstrap.cc:492] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: No bootstrap required, opened a new log
I20260619 22:21:23.957469 9357 ts_tablet_manager.cc:1397] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:23.957468 9358 raft_consensus.cc:348] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.957468 9361 raft_consensus.cc:348] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.957574 9358 raft_consensus.cc:374] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.957574 9361 raft_consensus.cc:374] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.957609 9361 raft_consensus.cc:729] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e295e176ea4f86a3045772ca8c753c, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.957609 9358 raft_consensus.cc:729] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b843f1f213e4f60a4751637ec8f7fea, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.957736 9361 consensus_queue.cc:260] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.957736 9358 consensus_queue.cc:260] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.957794 9357 raft_consensus.cc:348] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.957916 9357 raft_consensus.cc:374] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:23.957955 9357 raft_consensus.cc:729] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed51c42758a94de0b08bce0a6ac860e8, State: Initialized, Role: FOLLOWER
I20260619 22:21:23.957994 9361 ts_tablet_manager.cc:1428] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:21:23.957994 9358 ts_tablet_manager.cc:1428] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:21:23.958067 9357 consensus_queue.cc:260] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:23.958305 9357 ts_tablet_manager.cc:1428] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 22:21:24.217617 9365 raft_consensus.cc:482] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:24.217761 9365 raft_consensus.cc:504] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:24.218281 9365 leader_election.cc:288] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:24.218775 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496850bee9f348d484b2a797751c72a6" candidate_uuid: "f7e295e176ea4f86a3045772ca8c753c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea" is_pre_election: true
I20260619 22:21:24.218792 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496850bee9f348d484b2a797751c72a6" candidate_uuid: "f7e295e176ea4f86a3045772ca8c753c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed51c42758a94de0b08bce0a6ac860e8" is_pre_election: true
I20260619 22:21:24.218998 9283 raft_consensus.cc:2454] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e295e176ea4f86a3045772ca8c753c in term 0.
I20260619 22:21:24.219013 9003 raft_consensus.cc:2454] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e295e176ea4f86a3045772ca8c753c in term 0.
I20260619 22:21:24.219348 9074 leader_election.cc:302] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b843f1f213e4f60a4751637ec8f7fea, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:24.219532 9365 raft_consensus.cc:2781] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:24.219600 9365 raft_consensus.cc:482] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:24.219648 9365 raft_consensus.cc:3037] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:24.220621 9365 raft_consensus.cc:504] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:24.220952 9365 leader_election.cc:288] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [CANDIDATE]: Term 1 election: Requested vote from peers ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365), 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491)
I20260619 22:21:24.221272 9003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496850bee9f348d484b2a797751c72a6" candidate_uuid: "f7e295e176ea4f86a3045772ca8c753c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed51c42758a94de0b08bce0a6ac860e8"
I20260619 22:21:24.221331 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "496850bee9f348d484b2a797751c72a6" candidate_uuid: "f7e295e176ea4f86a3045772ca8c753c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b843f1f213e4f60a4751637ec8f7fea"
I20260619 22:21:24.221415 9003 raft_consensus.cc:3037] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:24.221498 9283 raft_consensus.cc:3037] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:24.222410 9283 raft_consensus.cc:2454] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e295e176ea4f86a3045772ca8c753c in term 1.
I20260619 22:21:24.222422 9003 raft_consensus.cc:2454] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e295e176ea4f86a3045772ca8c753c in term 1.
I20260619 22:21:24.222695 9074 leader_election.cc:302] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9b843f1f213e4f60a4751637ec8f7fea, ed51c42758a94de0b08bce0a6ac860e8, f7e295e176ea4f86a3045772ca8c753c; no voters:
I20260619 22:21:24.222929 9365 raft_consensus.cc:2781] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:24.223119 9365 raft_consensus.cc:686] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 1 LEADER]: Becoming Leader. State: Replica: f7e295e176ea4f86a3045772ca8c753c, State: Running, Role: LEADER
I20260619 22:21:24.223384 9365 consensus_queue.cc:237] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } }
I20260619 22:21:24.225781 8846 catalog_manager.cc:5495] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c reported cstate change: term changed from 0 to 1, leader changed from <none> to f7e295e176ea4f86a3045772ca8c753c (127.1.74.130). New cstate: current_term: 1 leader_uuid: "f7e295e176ea4f86a3045772ca8c753c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e295e176ea4f86a3045772ca8c753c" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 38061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 } health_report { overall_health: UNKNOWN } } }
I20260619 22:21:24.266731 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:24.263177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.1.74.130:38061)
0619 22:21:24.263447 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:24.263451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:24.263463 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:24.263530 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:24.263532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:24.263619 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:24.263721 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:24.263965 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.264473 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.265340 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.265583 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.265672 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:24.265899 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:24.266356 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:24.266443 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:24.266553 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 22:21:24.266600 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":118,"threads_started":1}
I20260619 22:21:24.277087 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:24.273697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:34283 (local address 127.1.74.131:37491)
0619 22:21:24.273843 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:24.273848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:24.274006 (+ 158us) server_negotiation.cc:407] Connection header received
0619 22:21:24.274181 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:24.274183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:24.274226 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:24.274307 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:24.274564 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.275179 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.276039 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.276259 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.276367 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:24.276646 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:24.276719 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:24.276798 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:24.276892 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 22:21:24.276940 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:21:24.277331 9447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:24.273528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37491 (local address 127.1.74.130:34283)
0619 22:21:24.273911 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:24.273929 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:24.274055 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:24.274317 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:24.274320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:24.274328 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:21:24.274461 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:24.274466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.275289 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.275293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:24.275920 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:24.275927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.276370 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.276373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:24.276466 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:24.276512 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:21:24.276805 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:21:24.276814 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 22:21:24.276874 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 22:21:24.276952 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":111,"threads_started":1}
I20260619 22:21:24.283075 9283 raft_consensus.cc:1261] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Refusing update from remote peer f7e295e176ea4f86a3045772ca8c753c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:24.283099 9003 raft_consensus.cc:1261] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Refusing update from remote peer f7e295e176ea4f86a3045772ca8c753c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:24.283509 9440 consensus_queue.cc:1035] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed51c42758a94de0b08bce0a6ac860e8" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:24.283828 9365 consensus_queue.cc:1035] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b843f1f213e4f60a4751637ec8f7fea" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:24.301440 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:24.298062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:60103 (local address 127.1.74.130:38061)
0619 22:21:24.298225 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:24.298230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:24.298287 (+ 57us) server_negotiation.cc:407] Connection header received
0619 22:21:24.298454 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:24.298457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:24.298508 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:24.298574 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:24.298812 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.299360 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.300231 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.300505 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.300621 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:24.300929 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:24.301015 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:24.301092 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:24.301222 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 22:21:24.301293 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 22:21:24.301450 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:24.297959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:38061 (local address 127.1.74.131:60103)
0619 22:21:24.298212 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:24.298228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:24.298332 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:24.298585 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:24.298588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:24.298595 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 22:21:24.298692 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:24.298698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.299466 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.299468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:24.300115 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:24.300125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:24.300621 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:24.300625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:24.300721 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:24.300777 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 22:21:24.301113 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 22:21:24.301119 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 22:21:24.301202 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 22:21:24.301294 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260619 22:21:24.323890 8843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:24.324044 8843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:24.326293 8843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5b03c247f6643be8d7042d1bec6caac: Sending DeleteTablet for 3 replicas of tablet 496850bee9f348d484b2a797751c72a6
I20260619 22:21:24.327322 9263 tablet_service.cc:1514] Processing DeleteTablet for tablet 496850bee9f348d484b2a797751c72a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:24 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:54494
I20260619 22:21:24.327317 8983 tablet_service.cc:1514] Processing DeleteTablet for tablet 496850bee9f348d484b2a797751c72a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:24 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:32918
I20260619 22:21:24.327515 9381 tablet_replica.cc:330] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: stopping tablet replica
I20260619 22:21:24.327636 9381 raft_consensus.cc:2229] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:24.327672 9382 tablet_replica.cc:330] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: stopping tablet replica
I20260619 22:21:24.327777 9381 raft_consensus.cc:2258] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:24.327839 9382 raft_consensus.cc:2229] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:24.327993 9382 raft_consensus.cc:2258] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:24.328176 9123 tablet_service.cc:1514] Processing DeleteTablet for tablet 496850bee9f348d484b2a797751c72a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:24 UTC) from {username='oryx', principal='oryx/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:52214
I20260619 22:21:24.328531 9383 tablet_replica.cc:330] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: stopping tablet replica
I20260619 22:21:24.328639 9383 raft_consensus.cc:2229] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:24.328933 9383 raft_consensus.cc:2258] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:24.329707 9381 ts_tablet_manager.cc:1905] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:24.330094 9383 ts_tablet_manager.cc:1905] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:24.332162 9383 ts_tablet_manager.cc:1918] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:21:24.332221 9383 log.cc:1196] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/496850bee9f348d484b2a797751c72a6
I20260619 22:21:24.332360 9381 ts_tablet_manager.cc:1918] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:21:24.332422 9383 ts_tablet_manager.cc:1939] T 496850bee9f348d484b2a797751c72a6 P f7e295e176ea4f86a3045772ca8c753c: Deleting consensus metadata
I20260619 22:21:24.332435 9381 log.cc:1196] T 496850bee9f348d484b2a797751c72a6 P ed51c42758a94de0b08bce0a6ac860e8: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/496850bee9f348d484b2a797751c72a6
I20260619 22:21:24.332782 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 8918
I20260619 22:21:24.335389 9382 ts_tablet_manager.cc:1905] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:24.335412 8830 catalog_manager.cc:4841] TS f7e295e176ea4f86a3045772ca8c753c (127.1.74.130:38061): tablet 496850bee9f348d484b2a797751c72a6 (table test-table [id=f5be5164046a4b7682c0ee964e92b0dd]) successfully deleted
I20260619 22:21:24.337863 9382 ts_tablet_manager.cc:1918] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 22:21:24.337949 9382 log.cc:1196] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/496850bee9f348d484b2a797751c72a6
I20260619 22:21:24.338201 9382 ts_tablet_manager.cc:1939] T 496850bee9f348d484b2a797751c72a6 P 9b843f1f213e4f60a4751637ec8f7fea: Deleting consensus metadata
I20260619 22:21:24.339043 8830 catalog_manager.cc:4841] TS 9b843f1f213e4f60a4751637ec8f7fea (127.1.74.131:37491): tablet 496850bee9f348d484b2a797751c72a6 (table test-table [id=f5be5164046a4b7682c0ee964e92b0dd]) successfully deleted
W20260619 22:21:24.340517 8831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:39365 (error 108)
I20260619 22:21:24.341365 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9058
I20260619 22:21:24.341670 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:24.341256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:39365 (local address 127.0.0.1:32952)
0619 22:21:24.341465 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:24.341542 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:39365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260619 22:21:24.342067 8831 catalog_manager.cc:4584] TS ed51c42758a94de0b08bce0a6ac860e8 (127.1.74.129:39365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 496850bee9f348d484b2a797751c72a6: Network error: Client connection negotiation failed: client connection to 127.1.74.129:39365: connect: Connection refused (error 111)
I20260619 22:21:24.348017 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9198
I20260619 22:21:24.354683 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 8814
2026-06-19T22:21:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3563 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:24 dist-test-slave-k84h krb5kdc[9459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:24 dist-test-slave-k84h krb5kdc[9459](info): set up 2 sockets
Jun 19 22:21:24 dist-test-slave-k84h krb5kdc[9459](info): commencing operation
krb5kdc: starting...
W20260619 22:21:25.411582 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:25 dist-test-slave-k84h krb5kdc[9459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:25.533514 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:41905
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:41905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:25.645232 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:25.645583 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:25.645658 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:25.649750 9472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:25.649864 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:25.649904 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:25.649941 9472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:25.649977 9472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:25.655495 9472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:41905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:41905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:25.656643 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:25.657678 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:25.666352 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:25.666404 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:25.666529 9472 server_base.cc:1034] running on GCE node
W20260619 22:21:25.666352 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:25.667290 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:25.668390 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:25.669579 9472 hybrid_clock.cc:648] HybridClock initialized: now 1781907685669555 us; error 42 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T22:21:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1326 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:25 dist-test-slave-k84h krb5kdc[9485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:25 dist-test-slave-k84h krb5kdc[9485](info): set up 2 sockets
Jun 19 22:21:25 dist-test-slave-k84h krb5kdc[9485](info): commencing operation
krb5kdc: starting...
W20260619 22:21:26.723587 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:26 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:26.844151 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:41229
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:26.954973 9498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:26.955281 9498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:26.955323 9498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:26.959491 9498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:26.959563 9498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:26.959583 9498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:26.959600 9498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:26.959652 9498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:26.965215 9498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:41229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:26.966504 9498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:26.967607 9498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:26.976343 9498 server_base.cc:1034] running on GCE node
W20260619 22:21:26.976291 9504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:26.976315 9503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:26.976286 9506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:26.977159 9498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:26.978255 9498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:26.979434 9498 hybrid_clock.cc:648] HybridClock initialized: now 1781907686979421 us; error 28 us; skew 500 ppm
Jun 19 22:21:26 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:26.982072 9498 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:26.982959 9498 webserver.cc:460] Webserver started at http://127.1.74.190:45209/ using document root <none> and password file <none>
I20260619 22:21:26.983469 9498 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:26.983525 9498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:26.983729 9498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:26.985702 9498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "db8c98a5b3c6474c9244f390130f6a45"
format_stamp: "Formatted at 2026-06-19 22:21:26 on dist-test-slave-k84h"
server_key: "ce2f622a4363aca3adebf1354123ea34"
server_key_iv: "087bd5ae3dea1b06cdd504eda4d11909"
server_key_version: "encryptionkey@0"
I20260619 22:21:26.986217 9498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "db8c98a5b3c6474c9244f390130f6a45"
format_stamp: "Formatted at 2026-06-19 22:21:26 on dist-test-slave-k84h"
server_key: "ce2f622a4363aca3adebf1354123ea34"
server_key_iv: "087bd5ae3dea1b06cdd504eda4d11909"
server_key_version: "encryptionkey@0"
I20260619 22:21:26.989862 9498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260619 22:21:26.992296 9513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:26.993482 9498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:26.993606 9498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "db8c98a5b3c6474c9244f390130f6a45"
format_stamp: "Formatted at 2026-06-19 22:21:26 on dist-test-slave-k84h"
server_key: "ce2f622a4363aca3adebf1354123ea34"
server_key_iv: "087bd5ae3dea1b06cdd504eda4d11909"
server_key_version: "encryptionkey@0"
I20260619 22:21:26.993714 9498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:27.026079 9498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:27.029151 9498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:27.029348 9498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:27.036290 9498 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:41229
I20260619 22:21:27.036310 9565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:41229 every 8 connection(s)
I20260619 22:21:27.037336 9498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:27.040226 9566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:27.041088 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9498
I20260619 22:21:27.041213 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:27.041538 1322 external_mini_cluster.cc:1428] Setting key e40548006949868987c1db1f6b09c01e
I20260619 22:21:27.046785 9566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Bootstrap starting.
I20260619 22:21:27.049512 9566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:27.050390 9566 log.cc:824] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:27.052627 9566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: No bootstrap required, opened a new log
I20260619 22:21:27.056015 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.043007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36860 (local address 127.1.74.190:41229)
0619 22:21:27.043405 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:27.043425 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:27.043459 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:21:27.044101 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:27.044118 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:27.044443 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:27.044719 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:27.045615 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.046604 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.047736 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.048208 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.048339 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.051206 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:27.051212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:27.051226 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:27.051254 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:27.053149 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.053617 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.053630 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.053636 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.053754 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.054041 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.054044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.054046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.054457 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:27.054663 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:27.054981 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 22:21:27.055284 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":125,"threads_started":1}
I20260619 22:21:27.056432 9566 raft_consensus.cc:348] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:27.056793 9566 raft_consensus.cc:374] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:27.056887 9566 raft_consensus.cc:729] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c98a5b3c6474c9244f390130f6a45, State: Initialized, Role: FOLLOWER
I20260619 22:21:27.057394 9566 consensus_queue.cc:260] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:27.057551 9566 raft_consensus.cc:388] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:27.057638 9566 raft_consensus.cc:482] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:27.057739 9566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:27.058890 9566 raft_consensus.cc:504] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:27.059247 9566 leader_election.cc:302] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db8c98a5b3c6474c9244f390130f6a45; no voters:
I20260619 22:21:27.059564 9566 leader_election.cc:288] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:27.059733 9571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:27.059994 9571 raft_consensus.cc:686] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 LEADER]: Becoming Leader. State: Replica: db8c98a5b3c6474c9244f390130f6a45, State: Running, Role: LEADER
I20260619 22:21:27.060359 9571 consensus_queue.cc:237] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:27.060823 9566 sys_catalog.cc:564] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:27.062135 9573 sys_catalog.cc:455] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db8c98a5b3c6474c9244f390130f6a45. Latest consensus state: current_term: 1 leader_uuid: "db8c98a5b3c6474c9244f390130f6a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } } }
I20260619 22:21:27.062130 9572 sys_catalog.cc:455] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db8c98a5b3c6474c9244f390130f6a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } } }
I20260619 22:21:27.062284 9572 sys_catalog.cc:458] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:27.062284 9573 sys_catalog.cc:458] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:27.063416 9581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:27.066526 9581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:27.072041 9581 catalog_manager.cc:1347] Generated new cluster ID: 2d37e43b7b2244e5820094f2f55ad0d5
I20260619 22:21:27.072130 9581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:27.087658 9581 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:27.089063 9581 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:27.103868 9581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Generated new TSK 0
I20260619 22:21:27.104852 9581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:27.137840 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:41229
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:27.261121 9592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:27.261492 9592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:27.261575 9592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:27.266510 9592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:27.266703 9592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:27.266793 9592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:27.272742 9592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:27.273918 9592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:27.275080 9592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:27.285097 9597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:27.285076 9600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:27.285066 9598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:27.285192 9592 server_base.cc:1034] running on GCE node
I20260619 22:21:27.285832 9592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:27.286551 9592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:27.287744 9592 hybrid_clock.cc:648] HybridClock initialized: now 1781907687287725 us; error 36 us; skew 500 ppm
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907687, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:27.290577 9592 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:27.291496 9592 webserver.cc:460] Webserver started at http://127.1.74.129:42973/ using document root <none> and password file <none>
I20260619 22:21:27.292016 9592 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:27.292073 9592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:27.292253 9592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:27.294240 9592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "222e99f8f04a4154b9c215a9f5133efa"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "3c7975d039880b610b0beb98f0e728da"
server_key_iv: "001a423640104342f6f4689f847c0051"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.294749 9592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "222e99f8f04a4154b9c215a9f5133efa"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "3c7975d039880b610b0beb98f0e728da"
server_key_iv: "001a423640104342f6f4689f847c0051"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.298602 9592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 22:21:27.301162 9607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.302445 9592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:27.302649 9592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "222e99f8f04a4154b9c215a9f5133efa"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "3c7975d039880b610b0beb98f0e728da"
server_key_iv: "001a423640104342f6f4689f847c0051"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.302793 9592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:27.326337 9592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:27.329907 9592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:27.330068 9592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:27.330602 9592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:27.331506 9592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:27.331581 9592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.331666 9592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:27.331712 9592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.341675 9592 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45173
I20260619 22:21:27.341714 9720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45173 every 8 connection(s)
I20260619 22:21:27.342662 9592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:27.345367 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9592
I20260619 22:21:27.345520 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:27.345847 1322 external_mini_cluster.cc:1428] Setting key 16535ffa13a2214b2121c1b2dacd02f0
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907687, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:27.358213 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.344635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:43061 (local address 127.1.74.190:41229)
0619 22:21:27.344778 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:27.344781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:27.345696 (+ 915us) server_negotiation.cc:407] Connection header received
0619 22:21:27.347263 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:27.347266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:27.347304 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:27.347395 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:27.348566 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.349279 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.350731 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.351031 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.351126 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.354338 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:27.354341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:27.354343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:27.354370 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:27.355938 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.356600 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.356603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.356605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.356681 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.357005 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.357007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.357008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.357265 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:27.357378 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:27.357913 (+ 535us) server_negotiation.cc:299] Negotiation successful
0619 22:21:27.358042 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:21:27.359323 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.345033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:43061)
0619 22:21:27.345519 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:27.345575 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:27.347044 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:27.347556 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:27.347562 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:27.347926 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:27.348412 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:27.348425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.349430 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.349436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:27.350571 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:27.350581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.351191 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.351198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:27.351446 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.352151 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:27.352176 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:27.354128 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:27.356061 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:27.356068 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:27.356085 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:27.356490 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:27.356779 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:27.356781 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:27.356783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:27.356923 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:27.357380 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:27.357390 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:27.357663 (+ 273us) client_negotiation.cc:769] Sending connection context
0619 22:21:27.357871 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 22:21:27.358298 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":139,"threads_started":1}
I20260619 22:21:27.360803 9721 heartbeater.cc:347] Connected to a master server at 127.1.74.190:41229
I20260619 22:21:27.361076 9721 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:27.361749 9721 heartbeater.cc:511] Master 127.1.74.190:41229 requested a full tablet report, sending...
I20260619 22:21:27.363552 9530 ts_manager.cc:194] Registered new tserver with Master: 222e99f8f04a4154b9c215a9f5133efa (127.1.74.129:45173)
I20260619 22:21:27.365329 9530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:43061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:21:27.378165 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:41229
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:27.490742 9726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:27.491039 9726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:27.491083 9726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:27.495327 9726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:27.495398 9726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:27.495460 9726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:27.501144 9726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:27.502251 9726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:27.503355 9726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:27.513520 9731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:27.513552 9732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:27.513520 9734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:27.513959 9726 server_base.cc:1034] running on GCE node
I20260619 22:21:27.514386 9726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:27.515138 9726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:27.516302 9726 hybrid_clock.cc:648] HybridClock initialized: now 1781907687516270 us; error 50 us; skew 500 ppm
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907687, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:27.519187 9726 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:27.520139 9726 webserver.cc:460] Webserver started at http://127.1.74.130:42713/ using document root <none> and password file <none>
I20260619 22:21:27.520731 9726 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:27.520825 9726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:27.521064 9726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:27.523309 9726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "c9c85ab454c24fb4a591a872840ff27c"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "911d804fcfd2d01fead68f17464c4404"
server_key_iv: "7474d3473c21775ca682b0ada74cd165"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.523932 9726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "c9c85ab454c24fb4a591a872840ff27c"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "911d804fcfd2d01fead68f17464c4404"
server_key_iv: "7474d3473c21775ca682b0ada74cd165"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.527891 9726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:21:27.530400 9741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.531759 9726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:27.531909 9726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "c9c85ab454c24fb4a591a872840ff27c"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "911d804fcfd2d01fead68f17464c4404"
server_key_iv: "7474d3473c21775ca682b0ada74cd165"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.532030 9726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:27.552078 9726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:27.555305 9726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:27.555518 9726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:27.556116 9726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:27.556957 9726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:27.557029 9726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.557101 9726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:27.557153 9726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.567087 9726 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:41725
I20260619 22:21:27.567108 9854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:41725 every 8 connection(s)
I20260619 22:21:27.568075 9726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:27.575206 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9726
I20260619 22:21:27.575338 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:27.575659 1322 external_mini_cluster.cc:1428] Setting key bb37aa65e5f8fa35c0fca53d6c666e2e
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907687, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:27.583709 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.569980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:33541 (local address 127.1.74.190:41229)
0619 22:21:27.570130 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:27.570134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:27.571170 (+ 1036us) server_negotiation.cc:407] Connection header received
0619 22:21:27.572573 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:27.572576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:27.572638 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:27.572753 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:27.573960 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.574501 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.575862 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.576156 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.576268 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.579366 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:27.579373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:27.579378 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:27.579412 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:27.581188 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.581818 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.581825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.581828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.581929 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.582302 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.582306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.582309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.582612 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:27.582709 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:27.583396 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 22:21:27.583534 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 22:21:27.584628 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.570378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:33541)
0619 22:21:27.571017 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:27.571062 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:27.572354 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:27.572939 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:27.572946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:27.573323 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:27.573813 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:27.573826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.574636 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.574645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:27.575711 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:27.575728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.576347 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.576352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:27.576568 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.577341 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:27.577364 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:27.579176 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:27.581341 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:27.581347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:27.581362 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:27.581678 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:27.582028 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:27.582030 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:27.582031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:27.582190 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:27.582766 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:27.582772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:27.583120 (+ 348us) client_negotiation.cc:769] Sending connection context
0619 22:21:27.583337 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 22:21:27.583719 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":129,"threads_started":1}
I20260619 22:21:27.586191 9855 heartbeater.cc:347] Connected to a master server at 127.1.74.190:41229
I20260619 22:21:27.586462 9855 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:21:27.587095 9855 heartbeater.cc:511] Master 127.1.74.190:41229 requested a full tablet report, sending...
I20260619 22:21:27.588260 9530 ts_manager.cc:194] Registered new tserver with Master: c9c85ab454c24fb4a591a872840ff27c (127.1.74.130:41725)
I20260619 22:21:27.588868 9530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:33541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:27.606642 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:41229
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:27.718956 9860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:27.719221 9860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:27.719264 9860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:27.723387 9860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:27.723454 9860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:27.723567 9860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:27.728838 9860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:27.729904 9860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:27.730942 9860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:27.740404 9865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:27.740427 9868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:27.740399 9866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:27.740600 9860 server_base.cc:1034] running on GCE node
I20260619 22:21:27.741101 9860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:27.741773 9860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:27.742915 9860 hybrid_clock.cc:648] HybridClock initialized: now 1781907687742901 us; error 30 us; skew 500 ppm
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907687, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:27.745688 9860 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:27.746551 9860 webserver.cc:460] Webserver started at http://127.1.74.131:39607/ using document root <none> and password file <none>
I20260619 22:21:27.747180 9860 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:27.747272 9860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:27.747499 9860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:27.749473 9860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "d18d99d6b6304aa18955931eec547f86"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "0a3a7c0ef7125fd8c9032047e4ee6eb7"
server_key_iv: "022fc5d96aa082396a2d27270891a6e6"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.750023 9860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "d18d99d6b6304aa18955931eec547f86"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "0a3a7c0ef7125fd8c9032047e4ee6eb7"
server_key_iv: "022fc5d96aa082396a2d27270891a6e6"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.753654 9860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 22:21:27.756053 9875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.757201 9860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:27.757339 9860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "d18d99d6b6304aa18955931eec547f86"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "0a3a7c0ef7125fd8c9032047e4ee6eb7"
server_key_iv: "022fc5d96aa082396a2d27270891a6e6"
server_key_version: "encryptionkey@0"
I20260619 22:21:27.757448 9860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:27.788218 9860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:27.791775 9860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:27.792002 9860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:27.792621 9860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:27.793484 9860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:27.793556 9860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.793628 9860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:27.793675 9860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:27.804147 9860 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:33173
I20260619 22:21:27.804175 9988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:33173 every 8 connection(s)
I20260619 22:21:27.805189 9860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:27.814066 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9860
I20260619 22:21:27.814232 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:27.814533 1322 external_mini_cluster.cc:1428] Setting key 20105624dd3875f2e3290a6dcec4449d
Jun 19 22:21:27 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907687, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:27.820246 9569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.807144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:45545 (local address 127.1.74.190:41229)
0619 22:21:27.807300 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:27.807304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:27.808278 (+ 974us) server_negotiation.cc:407] Connection header received
0619 22:21:27.809718 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:27.809722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:27.809788 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:27.809897 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:27.811268 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.811821 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.813042 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.813278 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.813406 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.816541 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:27.816545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:27.816547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:27.816581 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:27.818067 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.818603 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.818607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.818609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.818698 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:27.819130 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:27.819135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:27.819136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:27.819334 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:27.819443 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:27.819961 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 22:21:27.820084 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 22:21:27.821112 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:27.807582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:45545)
0619 22:21:27.808105 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:27.808155 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:27.809468 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:27.810109 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:27.810117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:27.810513 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:27.811108 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:27.811125 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.811970 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.811979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:27.812894 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:27.812903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:27.813448 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:27.813453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:27.813641 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:27.814424 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:27.814442 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:27.816369 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:27.818210 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:27.818216 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:27.818233 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:27.818499 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:27.818801 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:27.818804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:27.818806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:27.819044 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:27.819443 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:27.819452 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:27.819717 (+ 265us) client_negotiation.cc:769] Sending connection context
0619 22:21:27.819912 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 22:21:27.820276 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260619 22:21:27.822242 9989 heartbeater.cc:347] Connected to a master server at 127.1.74.190:41229
I20260619 22:21:27.822527 9989 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:27.823134 9989 heartbeater.cc:511] Master 127.1.74.190:41229 requested a full tablet report, sending...
I20260619 22:21:27.824306 9530 ts_manager.cc:194] Registered new tserver with Master: d18d99d6b6304aa18955931eec547f86 (127.1.74.131:33173)
I20260619 22:21:27.824968 9530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:45545
I20260619 22:21:27.828682 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab.
Entry for principal oryx/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab.
WARNING: no policy specified for oryx/127.1.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab.
Entry for principal oryx/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab.
WARNING: no policy specified for oryx/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab.
Entry for principal oryx/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab.
I20260619 22:21:27.909499 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9592
I20260619 22:21:27.915133 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9726
I20260619 22:21:27.920488 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9860
I20260619 22:21:27.925876 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9498
I20260619 22:21:27.932586 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:41229
--webserver_interface=127.1.74.190
--webserver_port=45209
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:28.042748 9998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:28.043140 9998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:28.043226 9998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:28.047235 9998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:28.047327 9998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:28.047369 9998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:28.047405 9998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:28.047435 9998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:28.052721 9998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:41229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=45209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:28.053800 9998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:28.054832 9998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:28.063347 10003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:28.063320 10004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.063468 9998 server_base.cc:1034] running on GCE node
W20260619 22:21:28.063388 10006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.064162 9998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:28.065253 9998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:28.066407 9998 hybrid_clock.cc:648] HybridClock initialized: now 1781907688066373 us; error 50 us; skew 500 ppm
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907688, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:28.069293 9998 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:28.070091 9998 webserver.cc:460] Webserver started at http://127.1.74.190:45209/ using document root <none> and password file <none>
I20260619 22:21:28.070590 9998 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:28.070659 9998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:28.074406 9998 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:28.076478 10013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.077687 9998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:28.077823 9998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "db8c98a5b3c6474c9244f390130f6a45"
format_stamp: "Formatted at 2026-06-19 22:21:26 on dist-test-slave-k84h"
server_key: "ce2f622a4363aca3adebf1354123ea34"
server_key_iv: "087bd5ae3dea1b06cdd504eda4d11909"
server_key_version: "encryptionkey@0"
I20260619 22:21:28.078408 9998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:28.092263 9998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:28.095539 9998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:28.095755 9998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:28.102885 9998 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:41229
I20260619 22:21:28.102897 10065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:41229 every 8 connection(s)
I20260619 22:21:28.103979 9998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:28.108173 10066 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:21:28.108863 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 9998
I20260619 22:21:28.109298 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:45173
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:41229
--webserver_port=42973
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:21:28.110004 10066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Bootstrap starting.
I20260619 22:21:28.133381 10066 log.cc:824] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:28.140543 10066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:21:28.140945 10066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Bootstrap complete.
I20260619 22:21:28.145390 10066 raft_consensus.cc:348] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:28.145855 10066 raft_consensus.cc:729] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c98a5b3c6474c9244f390130f6a45, State: Initialized, Role: FOLLOWER
I20260619 22:21:28.146400 10066 consensus_queue.cc:260] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:28.146548 10066 raft_consensus.cc:388] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:28.146647 10066 raft_consensus.cc:482] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:28.146765 10066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:21:28.148842 10066 raft_consensus.cc:504] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:28.149200 10066 leader_election.cc:302] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db8c98a5b3c6474c9244f390130f6a45; no voters:
I20260619 22:21:28.149536 10066 leader_election.cc:288] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 22:21:28.149709 10070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 22:21:28.150022 10070 raft_consensus.cc:686] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [term 2 LEADER]: Becoming Leader. State: Replica: db8c98a5b3c6474c9244f390130f6a45, State: Running, Role: LEADER
I20260619 22:21:28.150429 10070 consensus_queue.cc:237] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } }
I20260619 22:21:28.150812 10066 sys_catalog.cc:564] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:28.151902 10071 sys_catalog.cc:455] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db8c98a5b3c6474c9244f390130f6a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } } }
I20260619 22:21:28.152025 10071 sys_catalog.cc:458] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:28.152814 10072 sys_catalog.cc:455] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db8c98a5b3c6474c9244f390130f6a45. Latest consensus state: current_term: 2 leader_uuid: "db8c98a5b3c6474c9244f390130f6a45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8c98a5b3c6474c9244f390130f6a45" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 41229 } } }
I20260619 22:21:28.153018 10072 sys_catalog.cc:458] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:28.155781 10086 catalog_manager.cc:1259] Loaded cluster ID: 2d37e43b7b2244e5820094f2f55ad0d5
I20260619 22:21:28.155860 10086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: loading cluster ID for follower catalog manager: success
I20260619 22:21:28.157609 10086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: acquiring CA information for follower catalog manager: success
I20260619 22:21:28.158769 10086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:21:28.158941 10079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:28.159823 10079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:28.160275 10079 catalog_manager.cc:1259] Loaded cluster ID: 2d37e43b7b2244e5820094f2f55ad0d5
I20260619 22:21:28.160336 10079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:28.160930 10079 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:28.161352 10079 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db8c98a5b3c6474c9244f390130f6a45: Loaded TSK: 0
I20260619 22:21:28.162068 10079 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 22:21:28.235922 10068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:28.236202 10068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:28.236250 10068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:28.240418 10068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:28.240491 10068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:28.240556 10068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:28.246079 10068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.129.keytab
--principal=oryx/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:45173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=42973
--tserver_master_addrs=127.1.74.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:28.247253 10068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:28.248303 10068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:28.258530 10092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:28.258430 10093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:28.258428 10095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.258806 10068 server_base.cc:1034] running on GCE node
I20260619 22:21:28.259595 10068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:28.260283 10068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:28.261483 10068 hybrid_clock.cc:648] HybridClock initialized: now 1781907688261455 us; error 49 us; skew 500 ppm
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907688, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:28.264618 10068 init.cc:377] Logged in from keytab as oryx/127.1.74.129@KRBTEST.COM (short username oryx)
I20260619 22:21:28.265657 10068 webserver.cc:460] Webserver started at http://127.1.74.129:42973/ using document root <none> and password file <none>
I20260619 22:21:28.266255 10068 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:28.266384 10068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:28.270771 10068 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:28.273088 10102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.274456 10068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 22:21:28.274595 10068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "222e99f8f04a4154b9c215a9f5133efa"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "3c7975d039880b610b0beb98f0e728da"
server_key_iv: "001a423640104342f6f4689f847c0051"
server_key_version: "encryptionkey@0"
I20260619 22:21:28.275285 10068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:28.302289 10068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:28.305521 10068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:28.305735 10068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:28.306334 10068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:28.307185 10068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:28.307257 10068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.307325 10068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:28.307381 10068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.316524 10068 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45173
I20260619 22:21:28.316553 10215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45173 every 8 connection(s)
I20260619 22:21:28.317574 10068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:28.327173 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10068
I20260619 22:21:28.327625 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:41725
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:41229
--webserver_port=42713
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 22:21:28.331779 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.319717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:42499 (local address 127.1.74.190:41229)
0619 22:21:28.320261 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.320283 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.320508 (+ 225us) server_negotiation.cc:407] Connection header received
0619 22:21:28.322080 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.322091 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.322410 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.322773 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.324048 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.325024 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.326213 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.326688 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.326944 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.331021 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:42499: BlockingRecv error: recv got EOF from 127.1.74.129:42499 (error 108)
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":124,"threads_started":1}
I20260619 22:21:28.331847 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.319790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:42499)
0619 22:21:28.320323 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.320388 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.321653 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.322864 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.322873 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.323371 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.323852 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.323867 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.325184 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.325189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.326072 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.326082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.326938 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.326943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.327147 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.327969 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.327987 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.330763 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":133,"threads_started":1}
W20260619 22:21:28.332595 10216 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:41229 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.74.190:41229: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:28.339159 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.333080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:59051)
0619 22:21:28.333294 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.333306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.333369 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.333641 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.333643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.333780 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.333876 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.333881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.335038 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.335041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.335579 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.335587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.336254 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.336256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.336325 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.336892 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.336903 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.339051 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134}
W20260619 22:21:28.339437 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.333215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:59051 (local address 127.1.74.190:41229)
0619 22:21:28.333368 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.333374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.333395 (+ 21us) server_negotiation.cc:407] Connection header received
0619 22:21:28.333448 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.333453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.333517 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.333641 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.333991 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.334897 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.335721 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.336133 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.336279 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.339263 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:59051: BlockingRecv error: recv got EOF from 127.1.74.129:59051 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:28.346695 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.339794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:43429)
0619 22:21:28.339901 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.339916 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.339986 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.340312 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.340314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.340436 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.340505 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.340509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.341519 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.341521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.342226 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.342256 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.342948 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.342953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.343095 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.343682 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.343697 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.346543 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 22:21:28.346993 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.339930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:43429 (local address 127.1.74.190:41229)
0619 22:21:28.340071 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.340078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.340096 (+ 18us) server_negotiation.cc:407] Connection header received
0619 22:21:28.340142 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.340145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.340194 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.340283 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.340616 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.341413 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.342409 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.342754 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.342879 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.346671 (+ 3792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:43429: BlockingRecv error: recv got EOF from 127.1.74.129:43429 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 22:21:28.346978 10216 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:21:28.446662 10220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:28.447005 10220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:28.447049 10220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:28.451296 10220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:28.451373 10220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:28.451438 10220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:28.457397 10220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.130.keytab
--principal=oryx/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:41725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=42713
--tserver_master_addrs=127.1.74.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:28.458541 10220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:28.459584 10220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:28.470003 10225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.470062 10220 server_base.cc:1034] running on GCE node
W20260619 22:21:28.470003 10228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:28.470003 10226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.470669 10220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:28.471431 10220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:28.472640 10220 hybrid_clock.cc:648] HybridClock initialized: now 1781907688472622 us; error 31 us; skew 500 ppm
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907688, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:28.475720 10220 init.cc:377] Logged in from keytab as oryx/127.1.74.130@KRBTEST.COM (short username oryx)
I20260619 22:21:28.476723 10220 webserver.cc:460] Webserver started at http://127.1.74.130:42713/ using document root <none> and password file <none>
I20260619 22:21:28.477346 10220 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:28.477471 10220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:28.481771 10220 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:28.484272 10235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.485601 10220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:21:28.485747 10220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "c9c85ab454c24fb4a591a872840ff27c"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "911d804fcfd2d01fead68f17464c4404"
server_key_iv: "7474d3473c21775ca682b0ada74cd165"
server_key_version: "encryptionkey@0"
I20260619 22:21:28.486353 10220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:28.509344 10220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:28.512591 10220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:28.512809 10220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:28.513423 10220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:28.514274 10220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:28.514344 10220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.514410 10220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:28.514457 10220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.523706 10220 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:41725
I20260619 22:21:28.523732 10348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:41725 every 8 connection(s)
I20260619 22:21:28.524775 10220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:28.525044 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10220
I20260619 22:21:28.525537 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:33173
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:41229
--webserver_port=39607
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 22:21:28.536897 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.526937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:47797 (local address 127.1.74.190:41229)
0619 22:21:28.527081 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.527086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.528147 (+ 1061us) server_negotiation.cc:407] Connection header received
0619 22:21:28.529413 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.529416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.529469 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.529585 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.530819 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.531409 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.532576 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.532813 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.532925 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.536762 (+ 3837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:47797: BlockingRecv error: recv got EOF from 127.1.74.130:47797 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 22:21:28.537566 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.527421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:47797)
0619 22:21:28.527953 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.528014 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.529217 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.529782 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.529789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.530163 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.530664 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.530678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.531548 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.531552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.532442 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.532451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.532982 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.532986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.533160 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.533843 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.533859 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.536622 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
W20260619 22:21:28.538056 10349 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:41229 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.74.190:41229: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:28.544382 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.538520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:43427)
0619 22:21:28.538686 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.538700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.538773 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.539067 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.539070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.539261 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.539378 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.539385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.540109 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.540112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.541052 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.541062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.541476 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.541480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.541576 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.542078 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.542092 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.544196 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260619 22:21:28.544466 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.538617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:43427 (local address 127.1.74.190:41229)
0619 22:21:28.538733 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.538736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.538749 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:21:28.538909 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.538911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.538953 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.539024 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.539496 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.540007 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.541157 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.541373 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.541450 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.544333 (+ 2883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:43427: BlockingRecv error: recv got EOF from 127.1.74.130:43427 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:28.550208 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.544978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:41327)
0619 22:21:28.545098 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.545114 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.545192 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.545428 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.545430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.545556 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.545628 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.545632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.546638 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.546641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.547215 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.547224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.547798 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.547800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.547857 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.548344 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.548355 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.550104 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 22:21:28.550467 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.545053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:41327 (local address 127.1.74.190:41229)
0619 22:21:28.545164 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.545168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.545183 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:28.545283 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.545285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.545328 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.545401 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.545729 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.546538 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.547322 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.547668 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.547780 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.550280 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:41327: BlockingRecv error: recv got EOF from 127.1.74.130:41327 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 22:21:28.550493 10349 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:21:28.646371 10351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:28.646648 10351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:28.646692 10351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:28.650724 10351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:28.650797 10351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:28.650902 10351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:28.656605 10351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/krb5kdc/oryx_127.1.74.131.keytab
--principal=oryx/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:33173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=39607
--tserver_master_addrs=127.1.74.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:28.657516 10351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:28.658468 10351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:28.668221 10360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:28.668262 10357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.668241 10351 server_base.cc:1034] running on GCE node
W20260619 22:21:28.668277 10358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:28.668916 10351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:28.669509 10351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:28.670704 10351 hybrid_clock.cc:648] HybridClock initialized: now 1781907688670690 us; error 24 us; skew 500 ppm
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907688, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:28.673398 10351 init.cc:377] Logged in from keytab as oryx/127.1.74.131@KRBTEST.COM (short username oryx)
I20260619 22:21:28.674440 10351 webserver.cc:460] Webserver started at http://127.1.74.131:39607/ using document root <none> and password file <none>
I20260619 22:21:28.674996 10351 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:28.675069 10351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:28.678560 10351 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:28.680588 10367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.681807 10351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:28.681895 10351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "d18d99d6b6304aa18955931eec547f86"
format_stamp: "Formatted at 2026-06-19 22:21:27 on dist-test-slave-k84h"
server_key: "0a3a7c0ef7125fd8c9032047e4ee6eb7"
server_key_iv: "022fc5d96aa082396a2d27270891a6e6"
server_key_version: "encryptionkey@0"
I20260619 22:21:28.682464 10351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:28.715982 10351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:28.719028 10351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:28.719187 10351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:28.719715 10351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:28.720477 10351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:28.720521 10351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.720561 10351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:28.720575 10351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:28.729777 10351 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:33173
I20260619 22:21:28.729799 10480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:33173 every 8 connection(s)
I20260619 22:21:28.730816 10351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:28.734123 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10351
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 22:21:28.742650 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.732773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:37789 (local address 127.1.74.190:41229)
0619 22:21:28.732933 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.732938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.733779 (+ 841us) server_negotiation.cc:407] Connection header received
0619 22:21:28.735539 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.735542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.735604 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.735715 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.736856 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.737356 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.738491 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.738743 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.738861 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.742536 (+ 3675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:37789: BlockingRecv error: recv got EOF from 127.1.74.131:37789 (error 108)
Metrics: {"server-negotiator.queue_time_us":66,"spinlock_wait_cycles":2944}
I20260619 22:21:28.743462 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.733107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:37789)
0619 22:21:28.733600 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.733644 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.735299 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.735896 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.735903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.736255 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.736719 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.736731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.737478 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.737481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.738366 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.738375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.738923 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.738927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.739088 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.739722 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.739746 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.742403 (+ 2657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
W20260619 22:21:28.744022 10481 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:41229 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.74.190:41229: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
I20260619 22:21:28.744062 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.735258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.1.74.190:41229)
0619 22:21:28.735504 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.735507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.735518 (+ 11us) server_negotiation.cc:407] Connection header received
0619 22:21:28.735551 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.735553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.735622 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.735739 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.736462 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.737001 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.737849 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.738074 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.738184 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.739850 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:28.739856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:28.739870 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:28.739894 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:28.741765 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:28.742259 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:28.742263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:28.742268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:28.742441 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:28.742736 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:28.742742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:28.742743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:28.743152 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:28.743295 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:28.743636 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 22:21:28.743898 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"spinlock_wait_cycles":3968,"thread_start_us":89,"threads_started":1}
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:28.749220 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.744463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:44837)
0619 22:21:28.744556 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.744571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.744645 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.744885 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.744887 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.745011 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.745097 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.745101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.745865 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.745868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.746500 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.746508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.747060 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.747063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.747134 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.747510 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.747521 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.749137 (+ 1616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 22:21:28.749404 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.744541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:44837 (local address 127.1.74.190:41229)
0619 22:21:28.744644 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.744648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.744660 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:28.744734 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.744736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.744776 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.744863 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.745209 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.745744 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.746633 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.746967 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.747047 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.749287 (+ 2240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:44837: BlockingRecv error: recv got EOF from 127.1.74.131:44837 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:28 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:28.754117 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:28.749769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:47403)
0619 22:21:28.749886 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:28.749902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:28.749978 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:28.750157 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:28.750159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:28.750271 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:28.750334 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.750338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.751068 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.751070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.751568 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:28.751575 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.752092 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.752095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:28.752149 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.752505 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:28.752515 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:28.754031 (+ 1516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 22:21:28.754339 10218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:28.749865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:47403 (local address 127.1.74.190:41229)
0619 22:21:28.749997 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:28.750000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:28.750010 (+ 10us) server_negotiation.cc:407] Connection header received
0619 22:21:28.750039 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:28.750041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:28.750071 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:28.750133 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:28.750421 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.750979 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.751665 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:28.751986 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:28.752088 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:28.754235 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:47403: BlockingRecv error: recv got EOF from 127.1.74.131:47403 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 22:21:28.754345 10481 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 22:21:29 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:29 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:29.354002 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:29.347701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:59495)
0619 22:21:29.348030 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:29.348047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:29.348185 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:29.348812 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:29.348814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:29.349058 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:29.349181 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:29.349187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.349957 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.349960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:29.350635 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:29.350644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.351117 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.351119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:29.351183 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:29.351766 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:29.351778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:29.353867 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
W20260619 22:21:29.354156 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:29.348292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:59495 (local address 127.1.74.190:41229)
0619 22:21:29.348530 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:29.348533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:29.348545 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:29.348610 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:29.348614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:29.348671 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:29.348773 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:29.349295 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.349836 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.350744 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.351019 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.351100 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:29.353989 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:59495: BlockingRecv error: recv got EOF from 127.1.74.129:59495 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
Jun 19 22:21:29 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:29 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:29.557013 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:29.551288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:39047)
0619 22:21:29.551541 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:29.551555 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:29.551651 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:29.551937 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:29.551940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:29.552192 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:29.552290 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:29.552296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.553094 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.553096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:29.553812 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:29.553821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.554284 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.554287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:29.554360 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:29.554985 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:29.554996 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:29.556889 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
W20260619 22:21:29.557230 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:29.551373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:39047 (local address 127.1.74.190:41229)
0619 22:21:29.551505 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:29.551509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:29.551604 (+ 95us) server_negotiation.cc:407] Connection header received
0619 22:21:29.551732 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:29.551749 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:29.551793 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:29.551900 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:29.552435 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.552948 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.553934 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.554172 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.554283 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:29.557033 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:39047: BlockingRecv error: recv got EOF from 127.1.74.130:39047 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 22:21:29 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:29 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:29.760473 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:29.755017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:47655)
0619 22:21:29.755272 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:29.755290 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:29.755414 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:29.755753 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:29.755756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:29.755960 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:29.756049 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:29.756055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.756831 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.756837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:29.757404 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:29.757412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.757872 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.757876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:29.757955 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:29.758478 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:29.758489 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:29.760350 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
W20260619 22:21:29.760699 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:29.755128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:47655 (local address 127.1.74.190:41229)
0619 22:21:29.755274 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:29.755281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:29.755383 (+ 102us) server_negotiation.cc:407] Connection header received
0619 22:21:29.755514 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:29.755516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:29.755575 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:29.755695 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:29.756182 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.756722 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.757526 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:29.757761 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:29.757870 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:29.760551 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:47655: BlockingRecv error: recv got EOF from 127.1.74.131:47655 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 22:21:30 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:30 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:30.360239 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:30.354873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:54119)
0619 22:21:30.355162 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:30.355179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:30.355287 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:30.355492 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:30.355493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:30.355666 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:30.355777 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:30.355782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.356579 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.356582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:30.357153 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:30.357161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.357609 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.357611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:30.357670 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:30.358227 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:30.358239 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:30.360087 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
W20260619 22:21:30.360386 10491 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:30.355019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:54119 (local address 127.1.74.190:41229)
0619 22:21:30.355283 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:30.355286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:30.355300 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:30.355360 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:30.355362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:30.355404 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:30.355479 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:30.355918 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.356473 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.357274 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.357509 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.357592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:30.360234 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:54119: BlockingRecv error: recv got EOF from 127.1.74.129:54119 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
Jun 19 22:21:30 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:30 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:30.563005 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:30.557836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:45031)
0619 22:21:30.558124 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:30.558138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:30.558238 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:30.558475 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:30.558477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:30.558656 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:30.558781 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:30.558787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.559523 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.559525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:30.560082 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:30.560089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.560505 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.560507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:30.560566 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:30.561057 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:30.561068 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:30.562801 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
W20260619 22:21:30.563118 10491 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:30.557896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:45031 (local address 127.1.74.190:41229)
0619 22:21:30.558039 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:30.558044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:30.558209 (+ 165us) server_negotiation.cc:407] Connection header received
0619 22:21:30.558317 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:30.558319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:30.558364 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:30.558431 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:30.558910 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.559432 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.560190 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.560409 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.560489 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:30.562999 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:45031: BlockingRecv error: recv got EOF from 127.1.74.130:45031 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 22:21:30 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:30 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:30.777359 10493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:30.761234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:59961)
0619 22:21:30.761537 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:30.761553 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:30.761653 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:30.761879 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:30.761881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:30.762057 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:30.762149 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:30.762154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.762920 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.762922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:30.763472 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:30.763479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.763881 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.763884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:30.763946 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:30.775022 (+ 11076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:30.775044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:30.777144 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
W20260619 22:21:30.777590 10491 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:30.761395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:59961 (local address 127.1.74.190:41229)
0619 22:21:30.761553 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:30.761557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:30.761604 (+ 47us) server_negotiation.cc:407] Connection header received
0619 22:21:30.761722 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:30.761724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:30.761770 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:30.761842 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:30.762258 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.762786 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.763571 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:30.763781 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:30.763859 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:30.777451 (+ 13592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:59961: BlockingRecv error: recv got EOF from 127.1.74.131:59961 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 22:21:31 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:31 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:31.366911 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:31.361267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:60623)
0619 22:21:31.361556 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:31.361573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:31.361688 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:31.361953 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:31.361955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:31.362131 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:31.362225 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:31.362230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.363254 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.363260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:31.363913 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:31.363923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.364366 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.364369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:31.364444 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:31.364948 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:31.364960 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:31.366692 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":113,"threads_started":1}
W20260619 22:21:31.366968 10495 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:31.361331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:60623 (local address 127.1.74.190:41229)
0619 22:21:31.361577 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:31.361581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:31.361636 (+ 55us) server_negotiation.cc:407] Connection header received
0619 22:21:31.361783 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:31.361785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:31.361828 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:31.361920 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:31.362325 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.363083 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.364026 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.364255 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.364363 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:31.366811 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:60623: BlockingRecv error: recv got EOF from 127.1.74.129:60623 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
Jun 19 22:21:31 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:31 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:31.570052 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:31.563992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:51961)
0619 22:21:31.564314 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:31.564329 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:31.564443 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:31.564737 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:31.564740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:31.564963 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:31.565070 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:31.565076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.565894 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.565897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:31.566461 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:31.566468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.566992 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.566995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:31.567068 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:31.567795 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:31.567807 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:31.569932 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":114,"threads_started":1}
W20260619 22:21:31.570303 10495 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:31.564081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:51961 (local address 127.1.74.190:41229)
0619 22:21:31.564244 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:31.564250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:31.564431 (+ 181us) server_negotiation.cc:407] Connection header received
0619 22:21:31.564528 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:31.564530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:31.564584 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:31.564662 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:31.565201 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.565748 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.566570 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.566809 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.566930 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:31.570117 (+ 3187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:51961: BlockingRecv error: recv got EOF from 127.1.74.130:51961 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 22:21:31 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:31 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:31.784493 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:31.778342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:49951)
0619 22:21:31.778714 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:31.778729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:31.778874 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:31.779152 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:31.779158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:31.779355 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:31.779478 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:31.779483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.780350 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.780352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:31.780900 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:31.780907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.781370 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.781376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:31.781444 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:31.782017 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:31.782028 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:31.784290 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":112,"threads_started":1}
W20260619 22:21:31.784565 10495 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:31.778465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49951 (local address 127.1.74.190:41229)
0619 22:21:31.778613 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:31.778617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:31.778805 (+ 188us) server_negotiation.cc:407] Connection header received
0619 22:21:31.778960 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:31.778962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:31.779007 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:31.779110 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:31.779633 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.780238 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.781015 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:31.781251 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:31.781343 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:31.784441 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:49951: BlockingRecv error: recv got EOF from 127.1.74.131:49951 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 22:21:32 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:32 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:32.373485 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:32.367791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:33299)
0619 22:21:32.368078 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:32.368094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:32.368198 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:32.368443 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:32.368445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:32.368618 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:32.368712 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:32.368717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.369785 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.369790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:32.370528 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:32.370539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.371049 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.371051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:32.371116 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:32.371592 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:32.371603 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:32.373370 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":116,"threads_started":1}
W20260619 22:21:32.373692 10499 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:32.367971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:33299 (local address 127.1.74.190:41229)
0619 22:21:32.368224 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:32.368227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:32.368242 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:32.368281 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:32.368283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:32.368324 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:32.368412 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:32.368863 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.369659 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.370657 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.370937 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.371029 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:32.373511 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:33299: BlockingRecv error: recv got EOF from 127.1.74.129:33299 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
Jun 19 22:21:32 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:32 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:32.576085 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:32.570905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:44359)
0619 22:21:32.571159 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:32.571172 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:32.571263 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:32.571468 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:32.571471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:32.571734 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:32.571845 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:32.571852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.572567 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.572569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:32.573113 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:32.573121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.573511 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.573514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:32.573571 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:32.574037 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:32.574047 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:32.575915 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
W20260619 22:21:32.576156 10499 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:32.570979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:44359 (local address 127.1.74.190:41229)
0619 22:21:32.571124 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:32.571127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:32.571240 (+ 113us) server_negotiation.cc:407] Connection header received
0619 22:21:32.571341 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:32.571343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:32.571386 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:32.571452 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:32.571957 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.572478 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.573211 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.573421 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.573494 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:32.576046 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:44359: BlockingRecv error: recv got EOF from 127.1.74.130:44359 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 22:21:32 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:32 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:32.790691 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:32.785368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:60081)
0619 22:21:32.785652 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:32.785665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:32.785766 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:32.786004 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:32.786006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:32.786175 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:32.786264 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:32.786268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.787193 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.787196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:32.787748 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:32.787756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.788176 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.788178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:32.788238 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:32.788728 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:32.788740 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:32.790576 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
W20260619 22:21:32.790886 10499 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:32.785432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:60081 (local address 127.1.74.190:41229)
0619 22:21:32.785595 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:32.785602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:32.785761 (+ 159us) server_negotiation.cc:407] Connection header received
0619 22:21:32.785847 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:32.785849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:32.785898 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:32.785971 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:32.786432 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.787080 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.787855 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:32.788077 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:32.788157 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:32.790724 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:60081: BlockingRecv error: recv got EOF from 127.1.74.131:60081 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 22:21:33 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:33 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:33.379622 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:33.374294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:40513)
0619 22:21:33.374580 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:33.374593 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:33.374688 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:33.375048 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:33.375051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:33.375289 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:33.375384 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:33.375390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.376140 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.376142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:33.376687 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:33.376693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.377110 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.377112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:33.377173 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:33.377670 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:33.377681 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:33.379505 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":97,"threads_started":1}
W20260619 22:21:33.379866 10503 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:33.374355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:40513 (local address 127.1.74.190:41229)
0619 22:21:33.374585 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:33.374590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:33.374644 (+ 54us) server_negotiation.cc:407] Connection header received
0619 22:21:33.374774 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:33.374776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:33.374820 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:33.374964 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:33.375502 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.376053 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.376787 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.377016 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.377094 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:33.379699 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:40513: BlockingRecv error: recv got EOF from 127.1.74.129:40513 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":58,"threads_started":1}
Jun 19 22:21:33 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:33 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:33.583648 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:33.576928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:60281)
0619 22:21:33.577198 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:33.577212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:33.577292 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:33.577530 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:33.577532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:33.577710 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:33.577802 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:33.577806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.578563 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.578569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:33.579222 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:33.579232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.579728 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.579731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:33.579791 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:33.580464 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:33.580485 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:33.583475 (+ 2990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
W20260619 22:21:33.583825 10503 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:33.577107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:60281 (local address 127.1.74.190:41229)
0619 22:21:33.577244 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:33.577248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:33.577260 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:33.577377 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:33.577380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:33.577423 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:33.577496 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:33.577901 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.578455 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.579378 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.579615 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.579695 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:33.583674 (+ 3979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:60281: BlockingRecv error: recv got EOF from 127.1.74.130:60281 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 22:21:33 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:33 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:33.796645 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:33.791570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:60797)
0619 22:21:33.791829 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:33.791843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:33.791927 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:33.792191 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:33.792193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:33.792371 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:33.792461 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:33.792466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.793186 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.793188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:33.793728 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:33.793735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.794142 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.794144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:33.794203 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:33.794687 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:33.794698 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:33.796532 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
W20260619 22:21:33.796819 10503 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:33.791681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:60797 (local address 127.1.74.190:41229)
0619 22:21:33.791872 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:33.791876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:33.791890 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:33.792028 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:33.792030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:33.792083 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:33.792149 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:33.792560 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.793096 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.793825 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:33.794048 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:33.794124 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:33.796660 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:60797: BlockingRecv error: recv got EOF from 127.1.74.131:60797 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 22:21:34 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:34 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:34.385711 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:34.380329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:47365)
0619 22:21:34.380580 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:34.380592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:34.380677 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:34.380895 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:34.380897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:34.381063 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:34.381151 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:34.381155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.381863 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.381866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:34.382545 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:34.382555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.383100 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.383103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:34.383199 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:34.383748 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:34.383758 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:34.385600 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":65,"threads_started":1}
W20260619 22:21:34.385939 10507 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:34.380412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:47365 (local address 127.1.74.190:41229)
0619 22:21:34.380642 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:34.380645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:34.380657 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:21:34.380751 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:34.380753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:34.380793 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:34.380864 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:34.381234 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.381768 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.382647 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.382965 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.383085 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:34.385797 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:47365: BlockingRecv error: recv got EOF from 127.1.74.129:47365 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
Jun 19 22:21:34 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:34 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:34.594545 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:34.584709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:45463)
0619 22:21:34.585047 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:34.585064 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:34.585180 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:34.585537 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:34.585542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:34.585742 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:34.585887 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:34.585895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.587047 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.587053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:34.587978 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:34.587991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.588576 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.588582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:34.588676 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:34.592320 (+ 3644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:34.592334 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:34.594409 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
W20260619 22:21:34.594926 10507 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:34.585129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:45463 (local address 127.1.74.190:41229)
0619 22:21:34.585267 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:34.585271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:34.585288 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:21:34.585338 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:34.585341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:34.585392 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:34.585518 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:34.586057 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.586934 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.588096 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.588439 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.588555 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:34.594711 (+ 6156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:45463: BlockingRecv error: recv got EOF from 127.1.74.130:45463 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 22:21:34 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:34 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:34.803043 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:34.797515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:51015)
0619 22:21:34.797783 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:34.797796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:34.797893 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:34.798137 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:34.798139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:34.798316 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:34.798429 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:34.798434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.799176 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.799178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:34.799741 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:34.799749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.800292 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.800295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:34.800389 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:34.800996 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:34.801010 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:34.802922 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
W20260619 22:21:34.803143 10507 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:34.797578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:51015 (local address 127.1.74.190:41229)
0619 22:21:34.797730 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:34.797734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:34.797867 (+ 133us) server_negotiation.cc:407] Connection header received
0619 22:21:34.797981 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:34.797983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:34.798030 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:34.798104 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:34.798532 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.799084 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.799846 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:34.800165 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:34.800264 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:34.803018 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:51015: BlockingRecv error: recv got EOF from 127.1.74.131:51015 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 19 22:21:35 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:35 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:35.392527 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:35.386597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:54469)
0619 22:21:35.386945 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:35.386979 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:35.387136 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:35.387484 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:35.387487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:35.387747 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:35.387872 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:35.387881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.388640 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.388642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:35.389197 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:35.389204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.389784 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.389786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:35.389845 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:35.390422 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:35.390434 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:35.392405 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":74,"threads_started":1}
W20260619 22:21:35.392680 10510 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:35.386619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:54469 (local address 127.1.74.190:41229)
0619 22:21:35.386946 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:35.386952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:35.387066 (+ 114us) server_negotiation.cc:407] Connection header received
0619 22:21:35.387267 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:35.387273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:35.387337 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:35.387429 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:35.387994 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.388549 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.389321 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.389655 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.389784 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:35.392545 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:54469: BlockingRecv error: recv got EOF from 127.1.74.129:54469 (error 108)
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":86,"threads_started":1}
Jun 19 22:21:35 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:35 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:35.600989 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:35.595526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:40645)
0619 22:21:35.595860 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:35.595875 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:35.595996 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:35.596222 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:35.596224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:35.596408 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:35.596499 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:35.596504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.597293 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.597298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:35.597877 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:35.597886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.598362 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.598366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:35.598442 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:35.599037 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:35.599050 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:35.600871 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":99,"threads_started":1}
W20260619 22:21:35.601122 10510 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:35.595594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:40645 (local address 127.1.74.190:41229)
0619 22:21:35.595759 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:35.595764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:35.595952 (+ 188us) server_negotiation.cc:407] Connection header received
0619 22:21:35.596067 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:35.596070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:35.596116 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:35.596186 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:35.596603 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.597163 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.598005 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.598242 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.598356 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:35.601000 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:40645: BlockingRecv error: recv got EOF from 127.1.74.130:40645 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 22:21:35 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:35 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:35.815107 10513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:35.803893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:60471)
0619 22:21:35.804221 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:35.804236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:35.804337 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:35.804589 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:35.804591 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:35.804776 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:35.804867 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:35.804872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.805592 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.805594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:35.806134 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:35.806140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.806542 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.806544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:35.806602 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:35.812935 (+ 6333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:35.812950 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:35.814977 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":111,"threads_started":1}
W20260619 22:21:35.815259 10510 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:35.803980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:60471 (local address 127.1.74.190:41229)
0619 22:21:35.804109 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:35.804113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:35.804339 (+ 226us) server_negotiation.cc:407] Connection header received
0619 22:21:35.804408 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:35.804411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:35.804472 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:35.804549 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:35.804971 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.805498 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.806232 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:35.806440 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:35.806522 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:35.815132 (+ 8610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:60471: BlockingRecv error: recv got EOF from 127.1.74.131:60471 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 22:21:36 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:36 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:36.399115 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:36.393436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:54997)
0619 22:21:36.393725 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:36.393739 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:36.393830 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:36.394098 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:36.394100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:36.394276 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:36.394369 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:36.394375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.395194 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.395196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:36.395751 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:36.395759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.396177 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.396180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:36.396240 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:36.396744 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:36.396755 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:36.398927 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":104,"threads_started":1}
W20260619 22:21:36.399176 10515 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:36.393628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:54997 (local address 127.1.74.190:41229)
0619 22:21:36.393874 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:36.393878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:36.393892 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:36.393931 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:36.393934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:36.393976 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:36.394067 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:36.394475 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.395096 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.395855 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.396080 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.396170 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:36.399046 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:54997: BlockingRecv error: recv got EOF from 127.1.74.129:54997 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
Jun 19 22:21:36 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:36 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:36.607432 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:36.601821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:45403)
0619 22:21:36.602104 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:36.602117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:36.602245 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:36.602518 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:36.602521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:36.602726 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:36.602811 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:36.602816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.603751 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.603756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:36.604313 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:36.604321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.604781 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.604785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:36.604869 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:36.605351 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:36.605362 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:36.607316 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
W20260619 22:21:36.607645 10515 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:36.601868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:45403 (local address 127.1.74.190:41229)
0619 22:21:36.602046 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:36.602052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:36.602185 (+ 133us) server_negotiation.cc:407] Connection header received
0619 22:21:36.602338 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:36.602342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:36.602388 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:36.602487 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:36.602946 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.603642 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.604431 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.604673 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.604780 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:36.607464 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:45403: BlockingRecv error: recv got EOF from 127.1.74.130:45403 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 22:21:36 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:36 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:36.821413 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:36.816085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:41527)
0619 22:21:36.816362 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:36.816375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:36.816477 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:36.816721 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:36.816724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:36.816898 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:36.817012 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:36.817017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.817733 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.817735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:36.818291 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:36.818299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.818693 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.818696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:36.818754 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:36.819354 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:36.819365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:36.821267 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
W20260619 22:21:36.821628 10515 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:36.816147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:41527 (local address 127.1.74.190:41229)
0619 22:21:36.816303 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:36.816306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:36.816469 (+ 163us) server_negotiation.cc:407] Connection header received
0619 22:21:36.816565 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:36.816568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:36.816620 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:36.816682 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:36.817114 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.817639 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.818389 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:36.818601 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:36.818676 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:36.821495 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:41527: BlockingRecv error: recv got EOF from 127.1.74.131:41527 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 22:21:37 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:37 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:37.406419 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:37.400112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:43673)
0619 22:21:37.400424 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:37.400444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:37.400560 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:37.400796 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:37.400798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:37.400996 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:37.401092 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:37.401097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.402063 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.402067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:37.402777 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:37.402788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.403521 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.403527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:37.403631 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:37.404149 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:37.404162 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:37.406277 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
W20260619 22:21:37.406581 10519 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:37.400285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:43673 (local address 127.1.74.190:41229)
0619 22:21:37.400562 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:37.400566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:37.400581 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:37.400642 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:37.400644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:37.400686 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:37.400759 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:37.401228 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.401897 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.403001 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.403343 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.403448 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:37.406439 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:43673: BlockingRecv error: recv got EOF from 127.1.74.129:43673 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
Jun 19 22:21:37 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:37 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:37.613888 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:37.608238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:49395)
0619 22:21:37.608551 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:37.608565 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:37.608665 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:37.608919 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:37.608921 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:37.609111 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:37.609257 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:37.609264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.610023 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.610025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:37.610598 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:37.610607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.611101 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.611105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:37.611180 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:37.611758 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:37.611805 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:37.613737 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":134,"threads_started":1}
W20260619 22:21:37.614022 10519 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:37.608341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:49395 (local address 127.1.74.190:41229)
0619 22:21:37.608490 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:37.608494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:37.608635 (+ 141us) server_negotiation.cc:407] Connection header received
0619 22:21:37.608744 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:37.608746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:37.608791 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:37.608863 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:37.609381 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.609913 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.610702 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.610948 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.611032 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:37.613890 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:49395: BlockingRecv error: recv got EOF from 127.1.74.130:49395 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 22:21:37 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:37 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:37.829767 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:37.822404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:41225)
0619 22:21:37.822780 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:37.822796 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:37.822948 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:37.823411 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:37.823416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:37.823714 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:37.823868 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:37.823876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.825067 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.825070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:37.825730 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:37.825741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.826347 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.826351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:37.826440 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:37.827145 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:37.827158 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:37.829616 (+ 2458us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
W20260619 22:21:37.829989 10519 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:37.822625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:41225 (local address 127.1.74.190:41229)
0619 22:21:37.822796 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:37.822800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:37.822911 (+ 111us) server_negotiation.cc:407] Connection header received
0619 22:21:37.823064 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:37.823067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:37.823140 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:37.823228 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:37.824154 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.824861 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.825919 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:37.826190 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:37.826300 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:37.829833 (+ 3533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:41225: BlockingRecv error: recv got EOF from 127.1.74.131:41225 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Jun 19 22:21:38 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:38 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:38.413815 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:38.407426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:49435)
0619 22:21:38.407960 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:38.407978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:38.408118 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:38.408430 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:38.408433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:38.408693 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:38.408834 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:38.408843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.409857 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.409860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:38.410410 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:38.410418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.410910 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.410913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:38.410981 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:38.411557 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:38.411571 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:38.413683 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":96,"threads_started":1}
W20260619 22:21:38.414201 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:38.407596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:49435 (local address 127.1.74.190:41229)
0619 22:21:38.407922 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:38.407925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:38.408099 (+ 174us) server_negotiation.cc:407] Connection header received
0619 22:21:38.408228 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:38.408235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:38.408303 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:38.408381 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:38.409000 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.409753 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.410522 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.410738 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.410820 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:38.413888 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:49435: BlockingRecv error: recv got EOF from 127.1.74.129:49435 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":137,"threads_started":1}
Jun 19 22:21:38 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:38 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:38.620574 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:38.615001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:41673)
0619 22:21:38.615269 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:38.615285 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:38.615403 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:38.615697 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:38.615699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:38.615902 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:38.616000 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:38.616005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.616806 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.616809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:38.617380 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:38.617390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.617821 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.617824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:38.617895 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:38.618502 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:38.618514 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:38.620453 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":108,"threads_started":1}
W20260619 22:21:38.620736 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:38.615044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:41673 (local address 127.1.74.190:41229)
0619 22:21:38.615192 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:38.615195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:38.615399 (+ 204us) server_negotiation.cc:407] Connection header received
0619 22:21:38.615492 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:38.615495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:38.615566 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:38.615631 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:38.616133 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.616685 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.617497 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.617706 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.617782 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:38.620609 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:41673: BlockingRecv error: recv got EOF from 127.1.74.130:41673 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 22:21:38 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:38 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:38.836073 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:38.830719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:49711)
0619 22:21:38.831062 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:38.831076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:38.831157 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:38.831417 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:38.831419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:38.831608 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:38.831704 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:38.831708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.832445 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.832448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:38.833013 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:38.833020 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.833431 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.833434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:38.833491 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:38.834082 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:38.834093 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:38.835955 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":154,"threads_started":1}
W20260619 22:21:38.836223 10523 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:38.830919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49711 (local address 127.1.74.190:41229)
0619 22:21:38.831100 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:38.831104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:38.831129 (+ 25us) server_negotiation.cc:407] Connection header received
0619 22:21:38.831256 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:38.831258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:38.831312 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:38.831390 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:38.831807 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.832349 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.833116 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:38.833337 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:38.833416 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:38.836092 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:49711: BlockingRecv error: recv got EOF from 127.1.74.131:49711 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Jun 19 22:21:39 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:39 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:39.420356 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:39.414668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:53313)
0619 22:21:39.414987 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:39.415001 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:39.415089 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:39.415327 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:39.415329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:39.415522 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:39.415618 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:39.415623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.416456 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.416458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:39.417025 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:39.417033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.417450 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.417452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:39.417528 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:39.418125 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:39.418138 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:39.420238 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
W20260619 22:21:39.420588 10527 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:39.414824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:53313 (local address 127.1.74.190:41229)
0619 22:21:39.415109 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:39.415112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:39.415125 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:21:39.415160 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:39.415162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:39.415203 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:39.415296 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:39.415734 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.416360 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.417123 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.417350 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.417427 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:39.420386 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:53313: BlockingRecv error: recv got EOF from 127.1.74.129:53313 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":97,"threads_started":1}
Jun 19 22:21:39 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:39 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:39.627566 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:39.621560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:53425)
0619 22:21:39.621830 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:39.621845 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:39.621977 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:39.622251 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:39.622253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:39.622442 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:39.622538 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:39.622543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.623474 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.623477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:39.624099 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:39.624108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.624628 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.624630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:39.624695 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:39.625271 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:39.625283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:39.627411 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
W20260619 22:21:39.628046 10527 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:39.621708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:53425 (local address 127.1.74.190:41229)
0619 22:21:39.621892 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:39.621896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:39.621928 (+ 32us) server_negotiation.cc:407] Connection header received
0619 22:21:39.622087 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:39.622089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:39.622137 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:39.622214 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:39.622657 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.623347 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.624272 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.624516 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.624609 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:39.627932 (+ 3323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:53425: BlockingRecv error: recv got EOF from 127.1.74.130:53425 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 19 22:21:39 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:39 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:39.842252 10529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:39.836936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:37449)
0619 22:21:39.837219 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:39.837233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:39.837325 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:39.837555 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:39.837557 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:39.837732 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:39.837822 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:39.837826 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.838545 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.838547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:39.839165 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:39.839173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.839583 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.839586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:39.839643 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:39.840182 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:39.840192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:39.842068 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":124,"threads_started":1}
W20260619 22:21:39.842329 10527 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:39.837012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:37449 (local address 127.1.74.190:41229)
0619 22:21:39.837180 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:39.837186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:39.837301 (+ 115us) server_negotiation.cc:407] Connection header received
0619 22:21:39.837403 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:39.837405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:39.837453 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:39.837523 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:39.837936 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.838454 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.839274 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:39.839489 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:39.839571 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:39.842198 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:37449: BlockingRecv error: recv got EOF from 127.1.74.131:37449 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 19 22:21:40 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:40 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:40.427260 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:40.421323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:59071)
0619 22:21:40.421623 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:40.421637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:40.421731 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:40.422036 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:40.422039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:40.422241 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:40.422354 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:40.422359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.423280 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.423284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:40.423891 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:40.423902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.424379 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.424385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:40.424448 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:40.425022 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:40.425034 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:40.427109 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":118,"threads_started":1}
W20260619 22:21:40.427487 10531 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:40.421477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:59071 (local address 127.1.74.190:41229)
0619 22:21:40.421719 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:40.421723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:40.421737 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:40.421820 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:40.421822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:40.421866 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:40.421943 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:40.422506 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.423152 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.424021 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.424272 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.424360 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:40.427305 (+ 2945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:59071: BlockingRecv error: recv got EOF from 127.1.74.129:59071 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
Jun 19 22:21:40 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:40 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:40.635268 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:40.628522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:42401)
0619 22:21:40.628956 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:40.628971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:40.629072 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:40.629531 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:40.629535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:40.629738 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:40.629841 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:40.629847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.631048 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.631051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:40.631611 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:40.631620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.632131 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.632134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:40.632200 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:40.632846 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:40.632858 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:40.635122 (+ 2264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":127,"threads_started":1}
W20260619 22:21:40.635493 10531 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:40.628802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:42401 (local address 127.1.74.190:41229)
0619 22:21:40.629019 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:40.629025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:40.629052 (+ 27us) server_negotiation.cc:407] Connection header received
0619 22:21:40.629258 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:40.629264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:40.629353 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:40.629450 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:40.629983 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.630894 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.631738 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.632015 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.632108 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:40.635339 (+ 3231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:42401: BlockingRecv error: recv got EOF from 127.1.74.130:42401 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Jun 19 22:21:40 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:40 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:40.853698 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:40.843112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:56691)
0619 22:21:40.843492 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:40.843517 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:40.843671 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:40.843972 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:40.843975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:40.844212 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:40.844305 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:40.844311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.845104 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.845110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:40.845682 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:40.845690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.846162 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.846166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:40.846257 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:40.851420 (+ 5163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:40.851432 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:40.853541 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
W20260619 22:21:40.853821 10531 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:40.843257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:56691 (local address 127.1.74.190:41229)
0619 22:21:40.843448 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:40.843454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:40.843606 (+ 152us) server_negotiation.cc:407] Connection header received
0619 22:21:40.843789 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:40.843792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:40.843854 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:40.843953 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:40.844433 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.844983 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.845810 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:40.846040 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:40.846163 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:40.853706 (+ 7543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:56691: BlockingRecv error: recv got EOF from 127.1.74.131:56691 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Jun 19 22:21:41 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:41 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:41.433641 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:41.428099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:47451)
0619 22:21:41.428417 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:41.428432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:41.428538 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:41.428816 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:41.428822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:41.429021 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:41.429122 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:41.429129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.429949 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.429951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:41.430506 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:41.430514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.430979 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.430981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:41.431042 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:41.431557 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:41.431568 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:41.433458 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":109,"threads_started":1}
W20260619 22:21:41.433708 10535 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:41.428254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:47451 (local address 127.1.74.190:41229)
0619 22:21:41.428513 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:41.428517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:41.428531 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:21:41.428608 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:41.428610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:41.428654 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:41.428734 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:41.429277 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.429834 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.430624 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.430860 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.430967 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:41.433579 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:47451: BlockingRecv error: recv got EOF from 127.1.74.129:47451 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
Jun 19 22:21:41 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:41 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:41.642740 10536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:41.636499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:50503)
0619 22:21:41.636834 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:41.636850 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:41.636950 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:41.637251 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:41.637257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:41.637528 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:41.637660 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:41.637671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.638736 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.638739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:41.639363 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:41.639372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.639894 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.639898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:41.639968 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:41.640570 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:41.640585 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:41.642599 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":114,"threads_started":1}
W20260619 22:21:41.642942 10535 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:41.636752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:50503 (local address 127.1.74.190:41229)
0619 22:21:41.636931 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:41.636936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:41.636954 (+ 18us) server_negotiation.cc:407] Connection header received
0619 22:21:41.637030 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:41.637033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:41.637087 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:41.637187 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:41.637822 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.638610 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.639513 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.639760 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.639857 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:41.642783 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:50503: BlockingRecv error: recv got EOF from 127.1.74.130:50503 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 22:21:41 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:41 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:41.860572 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:41.854591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:58109)
0619 22:21:41.854921 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:41.854935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:41.855038 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:41.855302 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:41.855305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:41.855500 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:41.855604 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:41.855610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.856425 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.856427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:41.857004 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:41.857012 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.857458 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.857461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:41.857519 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:41.858120 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:41.858131 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:41.860376 (+ 2245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
W20260619 22:21:41.860617 10535 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:41.854714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:58109 (local address 127.1.74.190:41229)
0619 22:21:41.854901 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:41.854905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:41.855024 (+ 119us) server_negotiation.cc:407] Connection header received
0619 22:21:41.855115 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:41.855117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:41.855163 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:41.855235 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:41.855774 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.856320 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.857133 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:41.857351 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:41.857446 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:41.860513 (+ 3067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:58109: BlockingRecv error: recv got EOF from 127.1.74.131:58109 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 22:21:42 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:42 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:42.439949 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:42.434475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:58747)
0619 22:21:42.434793 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:42.434807 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:42.434979 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:42.435287 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:42.435290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:42.435489 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:42.435585 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:42.435591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.436331 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.436334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:42.436919 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:42.436929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.437370 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.437373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:42.437443 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:42.437964 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:42.437976 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:42.439802 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":97,"threads_started":1}
W20260619 22:21:42.440093 10539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:42.434538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:58747 (local address 127.1.74.190:41229)
0619 22:21:42.434895 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:42.434899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:42.434914 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:21:42.435089 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:42.435092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:42.435140 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:42.435216 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:42.435689 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.436226 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.437023 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.437241 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.437317 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:42.439950 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:58747: BlockingRecv error: recv got EOF from 127.1.74.129:58747 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":69,"threads_started":1}
Jun 19 22:21:42 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:42 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:42.649395 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:42.643723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:32813)
0619 22:21:42.644060 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:42.644077 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:42.644239 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:42.644540 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:42.644543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:42.644765 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:42.644853 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:42.644859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.645644 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.645650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:42.646244 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:42.646252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.646715 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.646719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:42.646804 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:42.647359 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:42.647370 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:42.649214 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":90,"threads_started":1}
W20260619 22:21:42.649464 10539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:42.643874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:32813 (local address 127.1.74.190:41229)
0619 22:21:42.644061 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:42.644067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:42.644175 (+ 108us) server_negotiation.cc:407] Connection header received
0619 22:21:42.644348 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:42.644351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:42.644409 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:42.644515 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:42.644987 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.645522 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.646372 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.646601 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.646712 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:42.649344 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:32813: BlockingRecv error: recv got EOF from 127.1.74.130:32813 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 22:21:42 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:42 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.131@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:42.867533 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:42.861387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.131:49679)
0619 22:21:42.861846 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:42.861859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:42.862013 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:42.862287 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:42.862290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:42.862504 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:42.862589 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:42.862595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.863435 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.863440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:42.864024 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:42.864032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.864487 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.864491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:42.864568 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:42.865057 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:42.865068 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:42.867312 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":124,"threads_started":1}
W20260619 22:21:42.867616 10539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:42.861531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49679 (local address 127.1.74.190:41229)
0619 22:21:42.861791 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:42.861796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:42.861937 (+ 141us) server_negotiation.cc:407] Connection header received
0619 22:21:42.862119 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:42.862122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:42.862177 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:42.862266 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:42.862721 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.863321 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.864146 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:42.864373 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:42.864492 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:42.867489 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.131:49679: BlockingRecv error: recv got EOF from 127.1.74.131:49679 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
Jun 19 22:21:43 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:43 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.129@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:43.446357 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:43.440898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.129:59327)
0619 22:21:43.441189 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:43.441202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:43.441288 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:43.441493 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:43.441495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:43.441677 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:43.441769 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:43.441774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.442513 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.442515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:43.443210 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:43.443221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.443674 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.443677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:43.443746 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:43.444261 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:43.444273 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:43.446238 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
W20260619 22:21:43.446542 10543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:43.441035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:59327 (local address 127.1.74.190:41229)
0619 22:21:43.441301 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:43.441305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:43.441327 (+ 22us) server_negotiation.cc:407] Connection header received
0619 22:21:43.441363 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:43.441365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:43.441407 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:43.441481 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:43.441874 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.442413 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.443329 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.443551 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.443632 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:43.446382 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.129:59327: BlockingRecv error: recv got EOF from 127.1.74.129:59327 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
Jun 19 22:21:43 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 22:21:43 dist-test-slave-k84h krb5kdc[9485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.74.130@KRBTEST.COM for oryx/127.1.74.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 22:21:43.657148 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:43.650546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:41229 (local address 127.1.74.130:48427)
0619 22:21:43.650947 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:43.650966 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:43.651127 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:43.651451 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:43.651454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:43.651685 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:43.651825 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:43.651832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.652682 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.652689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:43.653271 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:43.653280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.653845 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.653878 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:43.653979 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:43.654708 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:43.654723 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:43.657030 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.74.190:41229: Server oryx/127.1.74.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":117,"threads_started":1}
W20260619 22:21:43.657361 10543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:43.650793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:48427 (local address 127.1.74.190:41229)
0619 22:21:43.650947 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:43.650954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:43.651061 (+ 107us) server_negotiation.cc:407] Connection header received
0619 22:21:43.651267 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:43.651270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:43.651326 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:43.651434 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:43.651951 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.652568 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.653402 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.653686 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.653792 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:43.657182 (+ 3390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.74.130:48427: BlockingRecv error: recv got EOF from 127.1.74.130:48427 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 22:21:43.738209 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10068
I20260619 22:21:43.743953 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10220
I20260619 22:21:43.749219 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10351
I20260619 22:21:43.754314 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 9998
2026-06-19T22:21:43Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18072 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T22:21:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:43Z Disabled control of system clock
I20260619 22:21:43.785264 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33941
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:33941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 22:21:43.900187 10548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:43.900524 10548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:43.900592 10548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:43.904729 10548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:43.904793 10548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:43.904808 10548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:43.904826 10548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:43.904841 10548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:43.909942 10548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:33941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:43.911412 10548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:43.912817 10548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:43.921773 10554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:43.921842 10548 server_base.cc:1034] running on GCE node
W20260619 22:21:43.921886 10553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:43.921787 10556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:43.922739 10548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:43.923964 10548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:43.925133 10548 hybrid_clock.cc:648] HybridClock initialized: now 1781907703925105 us; error 47 us; skew 500 ppm
I20260619 22:21:43.927403 10548 webserver.cc:460] Webserver started at http://127.1.74.190:40825/ using document root <none> and password file <none>
I20260619 22:21:43.928025 10548 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:43.928110 10548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:43.928380 10548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:43.930562 10548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "545fe26a6cec43a3aee7726159590ca3"
format_stamp: "Formatted at 2026-06-19 22:21:43 on dist-test-slave-k84h"
server_key: "ad7eb9fc52477c5288e052f2a34c0066"
server_key_iv: "1cbd94967d60e8e386da5251eb1769dd"
server_key_version: "encryptionkey@0"
I20260619 22:21:43.931221 10548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "545fe26a6cec43a3aee7726159590ca3"
format_stamp: "Formatted at 2026-06-19 22:21:43 on dist-test-slave-k84h"
server_key: "ad7eb9fc52477c5288e052f2a34c0066"
server_key_iv: "1cbd94967d60e8e386da5251eb1769dd"
server_key_version: "encryptionkey@0"
I20260619 22:21:43.935179 10548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:43.938064 10562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:43.939510 10548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:43.939662 10548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "545fe26a6cec43a3aee7726159590ca3"
format_stamp: "Formatted at 2026-06-19 22:21:43 on dist-test-slave-k84h"
server_key: "ad7eb9fc52477c5288e052f2a34c0066"
server_key_iv: "1cbd94967d60e8e386da5251eb1769dd"
server_key_version: "encryptionkey@0"
I20260619 22:21:43.939798 10548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:43.966265 10548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:43.967131 10548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:43.967344 10548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:43.974956 10548 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:33941
I20260619 22:21:43.974979 10614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:33941 every 8 connection(s)
I20260619 22:21:43.976055 10548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:43.979518 10615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:43.982044 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10548
I20260619 22:21:43.982184 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:43.982491 1322 external_mini_cluster.cc:1428] Setting key 875493d6786d5678a2ca78d889662a4c
I20260619 22:21:43.986033 10615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3: Bootstrap starting.
I20260619 22:21:43.988488 10615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:43.989282 10615 log.cc:824] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:43.991634 10615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3: No bootstrap required, opened a new log
I20260619 22:21:43.992142 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:43.983989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.1.74.190:33941)
0619 22:21:43.984694 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:43.984711 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:43.984768 (+ 57us) server_negotiation.cc:407] Connection header received
0619 22:21:43.985409 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:43.985426 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:43.985848 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:43.986211 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:43.986654 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.987703 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.988578 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:43.989108 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:43.989275 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:43.990250 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:43.990256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:43.990269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:43.990660 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:43.990716 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:43.990737 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:43.990822 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:43.991039 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 22:21:43.991335 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":147,"threads_started":1}
I20260619 22:21:43.996181 10615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fe26a6cec43a3aee7726159590ca3" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33941 } }
I20260619 22:21:43.996438 10615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:43.996482 10615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545fe26a6cec43a3aee7726159590ca3, State: Initialized, Role: FOLLOWER
I20260619 22:21:43.996958 10615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fe26a6cec43a3aee7726159590ca3" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33941 } }
I20260619 22:21:43.997093 10615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:43.997184 10615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:43.997287 10615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:43.998318 10615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fe26a6cec43a3aee7726159590ca3" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33941 } }
I20260619 22:21:43.998704 10615 leader_election.cc:302] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545fe26a6cec43a3aee7726159590ca3; no voters:
I20260619 22:21:43.999094 10615 leader_election.cc:288] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:43.999239 10620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:43.999506 10620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [term 1 LEADER]: Becoming Leader. State: Replica: 545fe26a6cec43a3aee7726159590ca3, State: Running, Role: LEADER
I20260619 22:21:43.999912 10620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fe26a6cec43a3aee7726159590ca3" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33941 } }
I20260619 22:21:44.000504 10615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:44.001638 10621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545fe26a6cec43a3aee7726159590ca3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fe26a6cec43a3aee7726159590ca3" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33941 } } }
I20260619 22:21:44.001783 10621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:44.002125 10622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545fe26a6cec43a3aee7726159590ca3. Latest consensus state: current_term: 1 leader_uuid: "545fe26a6cec43a3aee7726159590ca3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545fe26a6cec43a3aee7726159590ca3" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33941 } } }
I20260619 22:21:44.002295 10622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:44.002198 10629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:44.005841 10629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:44.011816 10629 catalog_manager.cc:1347] Generated new cluster ID: db18c6bbc73f48f29e4ec8f20c7c8d20
I20260619 22:21:44.011912 10629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:44.028245 10629 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:44.029600 10629 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:44.041704 10629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545fe26a6cec43a3aee7726159590ca3: Generated new TSK 0
I20260619 22:21:44.043208 10629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:44.047039 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33941
--builtin_ntp_servers=127.1.74.148:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 22:21:44.161190 10639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:44.161540 10639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:44.161607 10639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:44.165848 10639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:44.165948 10639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:44.166043 10639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:44.171082 10639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:44.172307 10639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:44.173763 10639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:44.184379 10647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:44.184358 10644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:44.184518 10639 server_base.cc:1034] running on GCE node
W20260619 22:21:44.184358 10645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:44.185165 10639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:44.185869 10639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:44.187024 10639 hybrid_clock.cc:648] HybridClock initialized: now 1781907704187009 us; error 33 us; skew 500 ppm
I20260619 22:21:44.188968 10639 webserver.cc:460] Webserver started at http://127.1.74.129:40005/ using document root <none> and password file <none>
I20260619 22:21:44.189580 10639 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:44.189646 10639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:44.189909 10639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:44.192201 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "244c1021c8144f9297dc915a0a0c5a44"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "708dc5347438a3f357bb403edb92df0a"
server_key_iv: "d9237de7d5bfbd76baf85cb19f4cd35e"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.192807 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "244c1021c8144f9297dc915a0a0c5a44"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "708dc5347438a3f357bb403edb92df0a"
server_key_iv: "d9237de7d5bfbd76baf85cb19f4cd35e"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.196959 10639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:21:44.199811 10653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.201246 10639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:44.201396 10639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "244c1021c8144f9297dc915a0a0c5a44"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "708dc5347438a3f357bb403edb92df0a"
server_key_iv: "d9237de7d5bfbd76baf85cb19f4cd35e"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.201536 10639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:44.220556 10639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:44.221269 10639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:44.221472 10639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:44.222056 10639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:44.223122 10639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:44.223194 10639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.223276 10639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:44.223320 10639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.232252 10639 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:39229
I20260619 22:21:44.232270 10766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:39229 every 8 connection(s)
I20260619 22:21:44.233163 10639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:44.233912 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10639
I20260619 22:21:44.234036 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:44.234400 1322 external_mini_cluster.cc:1428] Setting key 5aa7ef1e5e1289d97d916a14f1b8f520
I20260619 22:21:44.236533 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33941
--builtin_ntp_servers=127.1.74.148:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 22:21:44.242687 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.235243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:55699 (local address 127.1.74.190:33941)
0619 22:21:44.235387 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:44.235391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:44.236484 (+ 1093us) server_negotiation.cc:407] Connection header received
0619 22:21:44.237935 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:44.237938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:44.237990 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:44.238070 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:44.238974 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.239518 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.240749 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.240974 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.241086 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:44.242176 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:44.242179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:44.242182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:44.242195 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:44.242221 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:44.242229 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:44.242310 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:44.242470 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 22:21:44.242530 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:21:44.243685 10769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.235664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33941 (local address 127.1.74.129:55699)
0619 22:21:44.236344 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:44.236385 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:44.237715 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:44.238237 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:44.238245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:44.238326 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:44.238801 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:44.238812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.239662 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.239667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:44.240611 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:44.240630 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.241126 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.241130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:44.241295 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:44.241916 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:44.241918 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:44.241956 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:44.242019 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:44.242312 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:44.242323 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:44.242331 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 22:21:44.242459 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 22:21:44.242759 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":100,"threads_started":1}
I20260619 22:21:44.244942 10767 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33941
I20260619 22:21:44.245204 10767 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:44.245711 10767 heartbeater.cc:511] Master 127.1.74.190:33941 requested a full tablet report, sending...
I20260619 22:21:44.247231 10579 ts_manager.cc:194] Registered new tserver with Master: 244c1021c8144f9297dc915a0a0c5a44 (127.1.74.129:39229)
I20260619 22:21:44.248353 10579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.129:55699
W20260619 22:21:44.347934 10770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:44.348246 10770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:44.348317 10770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:44.352486 10770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:44.352576 10770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:44.352667 10770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:44.357484 10770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:44.358628 10770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:44.360157 10770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:44.370407 10775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:44.370462 10770 server_base.cc:1034] running on GCE node
W20260619 22:21:44.370415 10778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:44.370407 10776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:44.371145 10770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:44.371819 10770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:44.372997 10770 hybrid_clock.cc:648] HybridClock initialized: now 1781907704372969 us; error 42 us; skew 500 ppm
I20260619 22:21:44.374905 10770 webserver.cc:460] Webserver started at http://127.1.74.130:46793/ using document root <none> and password file <none>
I20260619 22:21:44.375474 10770 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:44.375545 10770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:44.375711 10770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:44.377785 10770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "e755c34070e64d0f8e741956c08c68e8"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "ff9147b0d516f4ae4353af821b3b9b8a"
server_key_iv: "90ab4f8cc3d35ee93f01703780de2ab7"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.378335 10770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "e755c34070e64d0f8e741956c08c68e8"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "ff9147b0d516f4ae4353af821b3b9b8a"
server_key_iv: "90ab4f8cc3d35ee93f01703780de2ab7"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.382244 10770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:21:44.385030 10784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.386387 10770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:21:44.386507 10770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "e755c34070e64d0f8e741956c08c68e8"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "ff9147b0d516f4ae4353af821b3b9b8a"
server_key_iv: "90ab4f8cc3d35ee93f01703780de2ab7"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.386615 10770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:44.409370 10770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:44.410097 10770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:44.410271 10770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:44.410830 10770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:44.411913 10770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:44.411963 10770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.412006 10770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:44.412022 10770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.421092 10770 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:41263
I20260619 22:21:44.421113 10897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:41263 every 8 connection(s)
I20260619 22:21:44.422132 10770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:44.423097 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10770
I20260619 22:21:44.423218 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:44.423528 1322 external_mini_cluster.cc:1428] Setting key d5bb6d9aff3cde84697985a83111b1a0
I20260619 22:21:44.425766 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33941
--builtin_ntp_servers=127.1.74.148:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 22:21:44.434085 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.424387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:57877 (local address 127.1.74.190:33941)
0619 22:21:44.424542 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:44.424547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:44.425865 (+ 1318us) server_negotiation.cc:407] Connection header received
0619 22:21:44.427805 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:44.427809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:44.427862 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:44.427940 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:44.429053 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.429610 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.431731 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.431948 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.432034 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:44.433556 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:44.433559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:44.433561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:44.433576 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:44.433599 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:44.433607 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:44.433667 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:44.433889 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 22:21:44.433948 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:21:44.435689 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.424743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33941 (local address 127.1.74.130:57877)
0619 22:21:44.425674 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:44.425739 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:44.427493 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:44.428178 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:44.428188 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:44.428321 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:44.428896 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:44.428910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.429759 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.429766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:44.431570 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:44.431586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.432139 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.432146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:44.432413 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:44.433178 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:44.433182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:44.433255 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:44.433370 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:44.433701 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:44.433715 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:44.433723 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 22:21:44.433891 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 22:21:44.434336 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":152,"threads_started":1}
I20260619 22:21:44.437132 10898 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33941
I20260619 22:21:44.437440 10898 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:44.438194 10898 heartbeater.cc:511] Master 127.1.74.190:33941 requested a full tablet report, sending...
I20260619 22:21:44.439538 10579 ts_manager.cc:194] Registered new tserver with Master: e755c34070e64d0f8e741956c08c68e8 (127.1.74.130:41263)
I20260619 22:21:44.440148 10579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.130:57877
W20260619 22:21:44.545732 10901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:44.546028 10901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:44.546065 10901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:44.550240 10901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:44.550311 10901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:44.550395 10901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:44.555299 10901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:44.556419 10901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:44.557821 10901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:44.567663 10906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:44.567746 10909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:44.567672 10907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:44.568106 10901 server_base.cc:1034] running on GCE node
I20260619 22:21:44.568483 10901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:44.569052 10901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:44.570209 10901 hybrid_clock.cc:648] HybridClock initialized: now 1781907704570194 us; error 26 us; skew 500 ppm
I20260619 22:21:44.572060 10901 webserver.cc:460] Webserver started at http://127.1.74.131:33067/ using document root <none> and password file <none>
I20260619 22:21:44.572638 10901 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:44.572700 10901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:44.572950 10901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:44.575028 10901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "6a6be59aacd841f7b368776769db6d13"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "72f39cf44b5b3314e67e530d4eca43d5"
server_key_iv: "5a661b32da52552679854bae8766720f"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.575587 10901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "6a6be59aacd841f7b368776769db6d13"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "72f39cf44b5b3314e67e530d4eca43d5"
server_key_iv: "5a661b32da52552679854bae8766720f"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.579361 10901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:21:44.583613 10915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.585215 10901 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:44.585356 10901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "6a6be59aacd841f7b368776769db6d13"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "72f39cf44b5b3314e67e530d4eca43d5"
server_key_iv: "5a661b32da52552679854bae8766720f"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.585492 10901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:44.612488 10901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:44.613219 10901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:44.613418 10901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:44.614018 10901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:44.614991 10901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:44.615037 10901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.615113 10901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:44.615150 10901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.623636 10901 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:44991
I20260619 22:21:44.623730 11028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:44991 every 8 connection(s)
I20260619 22:21:44.624615 10901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:44.634301 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 10901
I20260619 22:21:44.634438 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.626587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:56585 (local address 127.1.74.190:33941)
0619 22:21:44.626721 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:44.626725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:44.627797 (+ 1072us) server_negotiation.cc:407] Connection header received
0619 22:21:44.629282 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:44.629286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:44.629350 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:44.629467 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:44.630476 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.631070 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.632316 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.632546 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.632677 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:44.633893 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:44.633896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:44.633899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:44.633913 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:44.633933 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:44.633940 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:44.634024 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:44.634205 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 22:21:44.634276 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:21:44.634474 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:44.634913 1322 external_mini_cluster.cc:1428] Setting key 58d9b6de6171193ecc54792764e069ff
I20260619 22:21:44.635701 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.626948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33941 (local address 127.1.74.131:56585)
0619 22:21:44.627633 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:44.627677 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:44.629041 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:44.629678 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:44.629686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:44.629786 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:44.630318 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:44.630332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.631219 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.631224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:44.632169 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:44.632179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:44.632740 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:44.632744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:44.632927 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:44.633587 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:44.633589 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:44.633634 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:44.633708 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:44.634042 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:44.634054 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:44.634064 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 22:21:44.634196 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 22:21:44.634527 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
I20260619 22:21:44.637027 11029 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33941
I20260619 22:21:44.637333 11029 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:44.637918 11029 heartbeater.cc:511] Master 127.1.74.190:33941 requested a full tablet report, sending...
I20260619 22:21:44.639091 10578 ts_manager.cc:194] Registered new tserver with Master: 6a6be59aacd841f7b368776769db6d13 (127.1.74.131:44991)
I20260619 22:21:44.639719 10578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.131:56585
I20260619 22:21:44.649334 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 22:21:44.652762 10618 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:44.651535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.1.74.190:33941)
0619 22:21:44.651670 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:44.651675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:44.651901 (+ 226us) server_negotiation.cc:407] Connection header received
0619 22:21:44.652016 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:44.652018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:44.652065 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:44.652144 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:44.652482 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40662: BlockingRecv error: recv got EOF from 127.0.0.1:40662 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:21:44.653417 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10639
I20260619 22:21:44.659621 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10770
I20260619 22:21:44.665606 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10901
I20260619 22:21:44.671389 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 10548
2026-06-19T22:21:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (916 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T22:21:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:44Z Disabled control of system clock
I20260619 22:21:44.702042 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42259
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:42259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 22:21:44.838095 11040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:44.838392 11040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:44.838430 11040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:44.842466 11040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:44.842533 11040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:44.842549 11040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:44.842566 11040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:44.842582 11040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:44.847359 11040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:42259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:44.848409 11040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:44.849845 11040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:44.858325 11048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:44.858351 11046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:44.858384 11045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:44.858537 11040 server_base.cc:1034] running on GCE node
I20260619 22:21:44.859261 11040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:44.860464 11040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:44.861641 11040 hybrid_clock.cc:648] HybridClock initialized: now 1781907704861613 us; error 46 us; skew 500 ppm
I20260619 22:21:44.863524 11040 webserver.cc:460] Webserver started at http://127.1.74.190:45129/ using document root <none> and password file <none>
I20260619 22:21:44.864111 11040 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:44.864202 11040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:44.864432 11040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:44.866460 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "f34aba233ad52bc404bfd0bd079ca99b"
server_key_iv: "e43b2ef27c57738a1519ca7f6409c920"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.867077 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "f34aba233ad52bc404bfd0bd079ca99b"
server_key_iv: "e43b2ef27c57738a1519ca7f6409c920"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.870803 11040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:44.873499 11054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:44.875053 11040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:44.875204 11040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff"
format_stamp: "Formatted at 2026-06-19 22:21:44 on dist-test-slave-k84h"
server_key: "f34aba233ad52bc404bfd0bd079ca99b"
server_key_iv: "e43b2ef27c57738a1519ca7f6409c920"
server_key_version: "encryptionkey@0"
I20260619 22:21:44.875347 11040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:44.880153 11040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:44.880831 11040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:44.881014 11040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:44.888058 11040 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:42259
I20260619 22:21:44.888069 11106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:42259 every 8 connection(s)
I20260619 22:21:44.889037 11040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:44.890920 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11040
I20260619 22:21:44.891047 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:44.891328 1322 external_mini_cluster.cc:1428] Setting key d960900910ff01ee2e95fa972db683b1
I20260619 22:21:44.892625 11107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:44.897117 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:44.892863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.1.74.190:42259)
0619 22:21:44.893567 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:44.893584 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:44.893622 (+ 38us) server_negotiation.cc:407] Connection header received
0619 22:21:44.894279 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:44.894297 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:44.894660 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:44.895049 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:44.895841 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:44.895845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:44.895859 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:44.895867 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:44.895918 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:44.895934 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:44.896092 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:44.896164 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 22:21:44.896411 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260619 22:21:44.898921 11107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff: Bootstrap starting.
I20260619 22:21:44.901228 11107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:44.902074 11107 log.cc:824] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:44.904057 11107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff: No bootstrap required, opened a new log
I20260619 22:21:44.907823 11107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42259 } }
I20260619 22:21:44.908049 11107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:44.908092 11107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ddfb7d9fb12491bbfb21e64b681b1ff, State: Initialized, Role: FOLLOWER
I20260619 22:21:44.908454 11107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42259 } }
I20260619 22:21:44.908563 11107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:44.908607 11107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:44.908680 11107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:44.909551 11107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42259 } }
I20260619 22:21:44.909853 11107 leader_election.cc:302] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ddfb7d9fb12491bbfb21e64b681b1ff; no voters:
I20260619 22:21:44.910112 11107 leader_election.cc:288] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:44.910244 11112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:44.910475 11112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [term 1 LEADER]: Becoming Leader. State: Replica: 4ddfb7d9fb12491bbfb21e64b681b1ff, State: Running, Role: LEADER
I20260619 22:21:44.910820 11112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42259 } }
I20260619 22:21:44.911437 11107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:44.912495 11114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ddfb7d9fb12491bbfb21e64b681b1ff. Latest consensus state: current_term: 1 leader_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42259 } } }
I20260619 22:21:44.912617 11114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:44.912935 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ddfb7d9fb12491bbfb21e64b681b1ff" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42259 } } }
I20260619 22:21:44.912993 11121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:44.913057 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:44.916299 11121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:44.921484 11121 catalog_manager.cc:1347] Generated new cluster ID: 8aedf3366a624d5a9cf1fba3de1cb657
I20260619 22:21:44.921554 11121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:44.940423 11121 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:44.940804 11121 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:44.950158 11121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ddfb7d9fb12491bbfb21e64b681b1ff: Generated new TSK 0
I20260619 22:21:44.950907 11121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:21:44.962931 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42259
--builtin_ntp_servers=127.1.74.148:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 22:21:45.073683 11131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:45.073945 11131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:45.073985 11131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:45.078083 11131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:45.078181 11131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:45.078274 11131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:45.083417 11131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:45.084466 11131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:45.085816 11131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:45.095409 11136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:45.095373 11137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:45.095396 11139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:45.095542 11131 server_base.cc:1034] running on GCE node
I20260619 22:21:45.096007 11131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:45.096714 11131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:45.097903 11131 hybrid_clock.cc:648] HybridClock initialized: now 1781907705097879 us; error 44 us; skew 500 ppm
I20260619 22:21:45.099777 11131 webserver.cc:460] Webserver started at http://127.1.74.129:44887/ using document root <none> and password file <none>
I20260619 22:21:45.100399 11131 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:45.100488 11131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:45.100719 11131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:45.102738 11131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "0cde8fd707e84bdca078f7e7e3623a5f"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "5e9756801d94c27011e782acadb84b75"
server_key_iv: "b601c7d62022e1cb67d76315c6fcc036"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.103354 11131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "0cde8fd707e84bdca078f7e7e3623a5f"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "5e9756801d94c27011e782acadb84b75"
server_key_iv: "b601c7d62022e1cb67d76315c6fcc036"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.106952 11131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:21:45.109611 11145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.111004 11131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:45.111146 11131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "0cde8fd707e84bdca078f7e7e3623a5f"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "5e9756801d94c27011e782acadb84b75"
server_key_iv: "b601c7d62022e1cb67d76315c6fcc036"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.111265 11131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:45.115891 11131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:45.116489 11131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:45.116675 11131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:45.117197 11131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:45.118055 11131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:45.118137 11131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.118207 11131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:45.118259 11131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.126741 11131 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:40605
I20260619 22:21:45.126765 11258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:40605 every 8 connection(s)
I20260619 22:21:45.127691 11131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:45.129242 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11131
I20260619 22:21:45.129364 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:45.129734 1322 external_mini_cluster.cc:1428] Setting key 74bd7caa37bee85a3bcda8868792615f
I20260619 22:21:45.131796 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42259
--builtin_ntp_servers=127.1.74.148:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 22:21:45.134337 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:45.129577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:49687 (local address 127.1.74.190:42259)
0619 22:21:45.129745 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:45.129750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:45.131129 (+ 1379us) server_negotiation.cc:407] Connection header received
0619 22:21:45.132593 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:45.132596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:45.132639 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:45.132718 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:45.133851 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:45.133854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:45.133857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:45.133869 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:45.133892 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:45.133901 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:45.134002 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:45.134157 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 22:21:45.134203 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 22:21:45.135296 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:45.130100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42259 (local address 127.1.74.129:49687)
0619 22:21:45.130939 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:45.131006 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:45.132375 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:45.132882 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:45.132889 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:45.132981 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:45.133586 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:45.133589 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:45.133632 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:45.133701 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:45.133997 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:45.134006 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:45.134014 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 22:21:45.134203 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 22:21:45.134447 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":163,"threads_started":1}
I20260619 22:21:45.136540 11259 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42259
I20260619 22:21:45.136798 11259 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:45.137214 11259 heartbeater.cc:511] Master 127.1.74.190:42259 requested a full tablet report, sending...
I20260619 22:21:45.138952 11071 ts_manager.cc:194] Registered new tserver with Master: 0cde8fd707e84bdca078f7e7e3623a5f (127.1.74.129:40605)
W20260619 22:21:45.246390 11262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:45.246666 11262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:45.246709 11262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:45.250756 11262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:45.250813 11262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:45.250888 11262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:45.255468 11262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:45.256502 11262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:45.257869 11262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:45.267742 11267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:45.267721 11270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:45.267726 11268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:45.267975 11262 server_base.cc:1034] running on GCE node
I20260619 22:21:45.268579 11262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:45.269229 11262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:45.270397 11262 hybrid_clock.cc:648] HybridClock initialized: now 1781907705270366 us; error 43 us; skew 500 ppm
I20260619 22:21:45.272302 11262 webserver.cc:460] Webserver started at http://127.1.74.130:33969/ using document root <none> and password file <none>
I20260619 22:21:45.272893 11262 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:45.272954 11262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:45.273170 11262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:45.275326 11262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "f8b003331e414b819965e27d10209ad7"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "408b7d1e07f6c8e4260713ba3f8f9592"
server_key_iv: "dcf82d49dce0dcf3f645214b702b82a5"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.275965 11262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "f8b003331e414b819965e27d10209ad7"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "408b7d1e07f6c8e4260713ba3f8f9592"
server_key_iv: "dcf82d49dce0dcf3f645214b702b82a5"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.279816 11262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:21:45.282418 11276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.283824 11262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:21:45.283972 11262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "f8b003331e414b819965e27d10209ad7"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "408b7d1e07f6c8e4260713ba3f8f9592"
server_key_iv: "dcf82d49dce0dcf3f645214b702b82a5"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.284101 11262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:45.288869 11262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:45.289516 11262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:45.289711 11262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:45.290268 11262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:45.291322 11262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:45.291394 11262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.291471 11262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:45.291517 11262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.300367 11262 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:45931
I20260619 22:21:45.300386 11389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:45931 every 8 connection(s)
I20260619 22:21:45.301324 11262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:45.307701 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:45.303227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:39861 (local address 127.1.74.190:42259)
0619 22:21:45.303361 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:45.303371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:45.304439 (+ 1068us) server_negotiation.cc:407] Connection header received
0619 22:21:45.305771 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:45.305774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:45.305821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:45.305904 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:45.307171 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:45.307175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:45.307178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:45.307191 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:45.307214 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:45.307231 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:45.307338 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:45.307469 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 22:21:45.307550 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:21:45.308482 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11262
I20260619 22:21:45.308643 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:45.308760 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:45.303607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42259 (local address 127.1.74.130:39861)
0619 22:21:45.304284 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:45.304326 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:45.305553 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:45.306077 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:45.306084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:45.306182 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:45.306816 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:45.306818 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:45.306933 (+ 115us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:45.307017 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:45.307315 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:45.307324 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:45.307329 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 22:21:45.307518 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 22:21:45.307893 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":114,"threads_started":1}
I20260619 22:21:45.309028 1322 external_mini_cluster.cc:1428] Setting key 6aa157342ddce2ce0c2d399015a5bfb8
I20260619 22:21:45.310110 11390 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42259
I20260619 22:21:45.310374 11390 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:45.310834 11390 heartbeater.cc:511] Master 127.1.74.190:42259 requested a full tablet report, sending...
I20260619 22:21:45.311399 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42259
--builtin_ntp_servers=127.1.74.148:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 22:21:45.312500 11071 ts_manager.cc:194] Registered new tserver with Master: f8b003331e414b819965e27d10209ad7 (127.1.74.130:45931)
W20260619 22:21:45.430423 11393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:45.430773 11393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:45.430816 11393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:45.435669 11393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:45.435760 11393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:45.435829 11393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:45.441321 11393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:42259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:45.442464 11393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:45.443920 11393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:45.454499 11398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:45.454591 11399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:45.454474 11401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:45.454788 11393 server_base.cc:1034] running on GCE node
I20260619 22:21:45.455272 11393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:45.456023 11393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:45.457250 11393 hybrid_clock.cc:648] HybridClock initialized: now 1781907705457197 us; error 69 us; skew 500 ppm
I20260619 22:21:45.459913 11393 webserver.cc:460] Webserver started at http://127.1.74.131:40559/ using document root <none> and password file <none>
I20260619 22:21:45.460592 11393 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:45.460698 11393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:45.460939 11393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:45.463297 11393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "622aeb215e8d49139a340ffdcf0c98d2"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "0296d39c41679cf0929ce8af5c44a70a"
server_key_iv: "bd91a410d5a5dfa3e3579ae78c853560"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.463912 11393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "622aeb215e8d49139a340ffdcf0c98d2"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "0296d39c41679cf0929ce8af5c44a70a"
server_key_iv: "bd91a410d5a5dfa3e3579ae78c853560"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.468816 11393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 22:21:45.473199 11407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.475188 11393 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:21:45.475373 11393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "622aeb215e8d49139a340ffdcf0c98d2"
format_stamp: "Formatted at 2026-06-19 22:21:45 on dist-test-slave-k84h"
server_key: "0296d39c41679cf0929ce8af5c44a70a"
server_key_iv: "bd91a410d5a5dfa3e3579ae78c853560"
server_key_version: "encryptionkey@0"
I20260619 22:21:45.475564 11393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:45.481160 11393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:45.481848 11393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:45.482061 11393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:45.482677 11393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:45.483808 11393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:45.483886 11393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.483969 11393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:45.484022 11393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:45.494019 11393 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:41715
I20260619 22:21:45.494040 11520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:41715 every 8 connection(s)
I20260619 22:21:45.495187 11393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:45.499610 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11393
I20260619 22:21:45.499841 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:45.500376 1322 external_mini_cluster.cc:1428] Setting key 28bcf9b66b4db6dab8b6c285766e8d20
I20260619 22:21:45.504456 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:45.497665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:59703 (local address 127.1.74.190:42259)
0619 22:21:45.497816 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:45.497820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:45.500084 (+ 2264us) server_negotiation.cc:407] Connection header received
0619 22:21:45.502024 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:45.502028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:45.502084 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:45.502209 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:45.503865 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:45.503869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:45.503874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:21:45.503892 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:45.503913 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:21:45.503941 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:45.504079 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:45.504259 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 22:21:45.504309 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:21:45.505574 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:45.498354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:42259 (local address 127.1.74.131:59703)
0619 22:21:45.499701 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:45.499892 (+ 191us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:45.501732 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:45.502497 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:45.502508 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:45.502718 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:45.503544 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:21:45.503547 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:45.503613 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:21:45.503703 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:45.504076 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:45.504086 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:45.504095 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 22:21:45.504317 (+ 222us) client_negotiation.cc:240] Negotiation successful
0619 22:21:45.504624 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":144,"threads_started":1}
I20260619 22:21:45.506949 11521 heartbeater.cc:347] Connected to a master server at 127.1.74.190:42259
I20260619 22:21:45.507344 11521 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:45.507833 11521 heartbeater.cc:511] Master 127.1.74.190:42259 requested a full tablet report, sending...
I20260619 22:21:45.508994 11071 ts_manager.cc:194] Registered new tserver with Master: 622aeb215e8d49139a340ffdcf0c98d2 (127.1.74.131:41715)
I20260619 22:21:45.515980 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 22:21:45.519351 11110 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:21:45.518430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.1.74.190:42259)
0619 22:21:45.518651 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:45.518657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:45.518675 (+ 18us) server_negotiation.cc:407] Connection header received
0619 22:21:45.518729 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:45.518734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:45.518779 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:45.518924 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:45.519207 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52912: BlockingRecv error: recv got EOF from 127.0.0.1:52912 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 22:21:45.519462 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11131
I20260619 22:21:45.525051 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11262
I20260619 22:21:45.530402 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11393
I20260619 22:21:45.535321 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11040
2026-06-19T22:21:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (862 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:45 dist-test-slave-k84h krb5kdc[11531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:45 dist-test-slave-k84h krb5kdc[11531](info): set up 2 sockets
Jun 19 22:21:45 dist-test-slave-k84h krb5kdc[11531](info): commencing operation
krb5kdc: starting...
W20260619 22:21:46.608445 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:46 dist-test-slave-k84h krb5kdc[11531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:21:46.747416 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33235
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:46.874400 11544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:46.874789 11544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:46.874926 11544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:46.879756 11544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:21:46.879887 11544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:46.879923 11544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:46.879977 11544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:21:46.880016 11544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:21:46.886049 11544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:46.887292 11544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:46.888441 11544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:21:46.898262 11544 server_base.cc:1034] running on GCE node
W20260619 22:21:46.898162 11549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:46.898128 11552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:46.898140 11550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:46.899493 11544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:46.901155 11544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:46.902488 11544 hybrid_clock.cc:648] HybridClock initialized: now 1781907706902416 us; error 92 us; skew 500 ppm
Jun 19 22:21:46 dist-test-slave-k84h krb5kdc[11531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:46.912896 11544 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:21:46.914110 11544 webserver.cc:460] Webserver started at http://127.1.74.190:42995/ using document root <none> and password file <none>
I20260619 22:21:46.914776 11544 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:46.914911 11544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:46.915180 11544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:46.917490 11544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "1ae2f682f2c84c04ba9d77da9faa19f6"
format_stamp: "Formatted at 2026-06-19 22:21:46 on dist-test-slave-k84h"
server_key: "d2258176e5e225b573b9c8c7edb622c6"
server_key_iv: "93f7ac1eda70011e631a60490d15d653"
server_key_version: "encryptionkey@0"
I20260619 22:21:46.918169 11544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "1ae2f682f2c84c04ba9d77da9faa19f6"
format_stamp: "Formatted at 2026-06-19 22:21:46 on dist-test-slave-k84h"
server_key: "d2258176e5e225b573b9c8c7edb622c6"
server_key_iv: "93f7ac1eda70011e631a60490d15d653"
server_key_version: "encryptionkey@0"
I20260619 22:21:46.922518 11544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:21:46.925786 11559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:46.927477 11544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:21:46.927681 11544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "1ae2f682f2c84c04ba9d77da9faa19f6"
format_stamp: "Formatted at 2026-06-19 22:21:46 on dist-test-slave-k84h"
server_key: "d2258176e5e225b573b9c8c7edb622c6"
server_key_iv: "93f7ac1eda70011e631a60490d15d653"
server_key_version: "encryptionkey@0"
I20260619 22:21:46.927841 11544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:46.944826 11544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:46.948662 11544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:46.948899 11544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:46.957007 11544 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:33235
I20260619 22:21:46.957401 11611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:33235 every 8 connection(s)
I20260619 22:21:46.958280 11544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:21:46.961673 11612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:46.965734 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11544
I20260619 22:21:46.965850 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:21:46.966183 1322 external_mini_cluster.cc:1428] Setting key f80fab5ccfc80f9f5993e2edc79c08ec
I20260619 22:21:46.968860 11612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: Bootstrap starting.
I20260619 22:21:46.971822 11612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:46.972628 11612 log.cc:824] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:46.975346 11612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: No bootstrap required, opened a new log
Jun 19 22:21:46 dist-test-slave-k84h krb5kdc[11531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:46.980132 11612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33235 } }
I20260619 22:21:46.980439 11612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:46.980525 11612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae2f682f2c84c04ba9d77da9faa19f6, State: Initialized, Role: FOLLOWER
I20260619 22:21:46.981184 11612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33235 } }
I20260619 22:21:46.981400 11612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:21:46.981501 11612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:21:46.981613 11612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:46.983057 11612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33235 } }
I20260619 22:21:46.983484 11612 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae2f682f2c84c04ba9d77da9faa19f6; no voters:
I20260619 22:21:46.983878 11612 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:21:46.984004 11617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:46.984300 11617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae2f682f2c84c04ba9d77da9faa19f6, State: Running, Role: LEADER
I20260619 22:21:46.984365 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:46.967967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.1.74.190:33235)
0619 22:21:46.968535 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:46.968563 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:46.968606 (+ 43us) server_negotiation.cc:407] Connection header received
0619 22:21:46.969393 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:46.969413 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:46.969882 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:46.970197 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:46.971205 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:46.972775 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:46.974028 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:46.974985 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:46.975126 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:46.978482 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:46.978489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:46.978505 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:46.978541 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:46.980420 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:46.981059 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:46.981067 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:46.981074 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:46.981246 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:46.981682 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:46.981688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:46.981690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:46.982281 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:46.982510 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:46.982870 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 22:21:46.983238 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":182,"threads_started":1}
I20260619 22:21:46.985091 11617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33235 } }
I20260619 22:21:46.985291 11612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:21:46.987460 11619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33235 } } }
I20260619 22:21:46.987599 11619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [sys.catalog]: This master's current role is: LEADER
I20260619 22:21:46.987970 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae2f682f2c84c04ba9d77da9faa19f6. Latest consensus state: current_term: 1 leader_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae2f682f2c84c04ba9d77da9faa19f6" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33235 } } }
I20260619 22:21:46.988346 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6 [sys.catalog]: This master's current role is: LEADER
W20260619 22:21:46.991102 11631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:21:46.991195 11631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:21:46.991318 11632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:21:46.992233 11632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:21:46.999349 11632 catalog_manager.cc:1347] Generated new cluster ID: 9c9ee9fa14204fb8adcb0ef4f898e3cf
I20260619 22:21:46.999452 11632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:21:47.014762 11632 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:21:47.016129 11632 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:21:47.023466 11632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: Generated new TSK 0
I20260619 22:21:47.024387 11632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:21:47.058447 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33235
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:47.184080 11638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:47.184448 11638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:47.184500 11638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:47.188990 11638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:47.189076 11638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:47.189165 11638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:21:47.195137 11638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:47.196314 11638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:47.197561 11638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:47.208950 11646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:47.209144 11638 server_base.cc:1034] running on GCE node
W20260619 22:21:47.208950 11643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:47.208950 11644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:47.209707 11638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:47.210456 11638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:47.211673 11638 hybrid_clock.cc:648] HybridClock initialized: now 1781907707211657 us; error 29 us; skew 500 ppm
Jun 19 22:21:47 dist-test-slave-k84h krb5kdc[11531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907707, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:47.214952 11638 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:21:47.215986 11638 webserver.cc:460] Webserver started at http://127.1.74.129:46879/ using document root <none> and password file <none>
I20260619 22:21:47.216552 11638 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:47.216615 11638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:47.216790 11638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:47.219027 11638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "b19e9294cf514a96b4cfc3bc3d1e7227"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "75ace28128704aea2c02afd1cda6b574"
server_key_iv: "1c0c14d2d72ed87558601eac002e91d8"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.219580 11638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "b19e9294cf514a96b4cfc3bc3d1e7227"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "75ace28128704aea2c02afd1cda6b574"
server_key_iv: "1c0c14d2d72ed87558601eac002e91d8"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.223816 11638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:21:47.226656 11653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.228121 11638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:21:47.228236 11638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "b19e9294cf514a96b4cfc3bc3d1e7227"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "75ace28128704aea2c02afd1cda6b574"
server_key_iv: "1c0c14d2d72ed87558601eac002e91d8"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.228339 11638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:47.249218 11638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:47.253000 11638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:47.253190 11638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:47.253777 11638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:47.254743 11638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:47.254801 11638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.254909 11638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:47.254947 11638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.265727 11638 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:44425
I20260619 22:21:47.265763 11766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:44425 every 8 connection(s)
I20260619 22:21:47.266789 11638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:21:47.276294 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11638
I20260619 22:21:47.276475 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:21:47.276927 1322 external_mini_cluster.cc:1428] Setting key 5f86c8ab025a60c0062885fbe78c9f5e
Jun 19 22:21:47 dist-test-slave-k84h krb5kdc[11531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907707, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:47.286710 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.268948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:44121 (local address 127.1.74.190:33235)
0619 22:21:47.269093 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.269102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.270194 (+ 1092us) server_negotiation.cc:407] Connection header received
0619 22:21:47.271794 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.271796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.271849 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.272029 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:47.273269 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.273948 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.275309 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.275548 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.275643 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.279468 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:47.279472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:47.279474 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:47.279509 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:47.283861 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.284518 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.284524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.284526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.284629 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.285078 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.285081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.285083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.285345 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:47.285456 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.286328 (+ 872us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.286490 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:21:47.287845 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.269244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33235 (local address 127.1.74.129:44121)
0619 22:21:47.270021 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.270069 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.271555 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.272168 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.272176 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.272580 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:47.273080 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.273095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.274103 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.274106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.275168 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.275179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.275755 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.275786 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.275974 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.276794 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:47.276811 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:47.279237 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:47.284038 (+ 4801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:47.284044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:47.284059 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:47.284373 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:47.284729 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:47.284731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:47.284732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:47.284961 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:47.285459 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:47.285465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:47.285738 (+ 273us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.285998 (+ 260us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.286623 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":119,"threads_started":1}
I20260619 22:21:47.289315 11767 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33235
I20260619 22:21:47.289572 11767 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:21:47.290154 11767 heartbeater.cc:511] Master 127.1.74.190:33235 requested a full tablet report, sending...
I20260619 22:21:47.291810 11576 ts_manager.cc:194] Registered new tserver with Master: b19e9294cf514a96b4cfc3bc3d1e7227 (127.1.74.129:44425)
I20260619 22:21:47.293334 11576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:44121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:21:47.305986 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33235
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:47.424954 11772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:47.425271 11772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:47.425369 11772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:47.429487 11772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:47.429584 11772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:47.429672 11772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:21:47.435333 11772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:47.436465 11772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:47.437489 11772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:47.447697 11780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:47.447697 11778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:47.447824 11772 server_base.cc:1034] running on GCE node
W20260619 22:21:47.447755 11777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:47.448369 11772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:47.449051 11772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:47.450243 11772 hybrid_clock.cc:648] HybridClock initialized: now 1781907707450226 us; error 29 us; skew 500 ppm
Jun 19 22:21:47 dist-test-slave-k84h krb5kdc[11531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907707, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:47.453389 11772 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:21:47.454378 11772 webserver.cc:460] Webserver started at http://127.1.74.130:36815/ using document root <none> and password file <none>
I20260619 22:21:47.455003 11772 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:47.455076 11772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:47.455276 11772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:47.457360 11772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "a0807e48d9b446658783af9697f23643"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "21b4af56a358ba0ca1b7e7523fe0d775"
server_key_iv: "81fba509f92d246e8c1f5420eb0fb643"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.457907 11772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "a0807e48d9b446658783af9697f23643"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "21b4af56a358ba0ca1b7e7523fe0d775"
server_key_iv: "81fba509f92d246e8c1f5420eb0fb643"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.461922 11772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260619 22:21:47.464711 11787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.466012 11772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:21:47.466166 11772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "a0807e48d9b446658783af9697f23643"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "21b4af56a358ba0ca1b7e7523fe0d775"
server_key_iv: "81fba509f92d246e8c1f5420eb0fb643"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.466305 11772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:47.479135 11772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:47.482548 11772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:47.482827 11772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:47.483546 11772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:47.484440 11772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:47.484517 11772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.484592 11772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:47.484647 11772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.494976 11772 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:37287
I20260619 22:21:47.494994 11900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:37287 every 8 connection(s)
I20260619 22:21:47.495944 11772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:21:47.503373 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11772
I20260619 22:21:47.503468 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:21:47.503783 1322 external_mini_cluster.cc:1428] Setting key 0b9e857c897290268b9dcd7815cafd5f
Jun 19 22:21:47 dist-test-slave-k84h krb5kdc[11531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907707, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:47.510641 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.497854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:50499 (local address 127.1.74.190:33235)
0619 22:21:47.498020 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.498024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.498993 (+ 969us) server_negotiation.cc:407] Connection header received
0619 22:21:47.500425 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.500429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.500487 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.500599 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:47.501816 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.502361 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.503703 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.503979 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.504089 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.506880 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:47.506887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:47.506891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:47.506923 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:47.508375 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.509048 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.509053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.509055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.509141 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.509507 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.509510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.509512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.509703 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:47.509814 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.510330 (+ 516us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.510465 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 22:21:47.511571 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.498284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33235 (local address 127.1.74.130:50499)
0619 22:21:47.498775 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.498820 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.500192 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.500781 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.500788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.501169 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:47.501650 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.501675 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.502495 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.502503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.503564 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.503575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.504136 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.504140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.504317 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.504919 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:47.504940 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:47.506669 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:47.508614 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:47.508621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:47.508640 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:47.508929 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:47.509245 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:47.509248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:47.509249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:47.509396 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:47.509810 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:47.509818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:47.510090 (+ 272us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.510292 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.510640 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":118,"threads_started":1}
I20260619 22:21:47.513111 11901 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33235
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:21:47.513433 11901 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:47.514000 11901 heartbeater.cc:511] Master 127.1.74.190:33235 requested a full tablet report, sending...
I20260619 22:21:47.515218 11576 ts_manager.cc:194] Registered new tserver with Master: a0807e48d9b446658783af9697f23643 (127.1.74.130:37287)
I20260619 22:21:47.515899 11576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:50499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:21:47.534190 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33235
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:21:47.668627 11906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:21:47.669029 11906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:21:47.669158 11906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:21:47.674122 11906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:21:47.674232 11906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:21:47.674332 11906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:21:47.680644 11906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:21:47.682010 11906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:21:47.683272 11906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:21:47.695401 11914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:21:47.695367 11912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:47.695470 11906 server_base.cc:1034] running on GCE node
W20260619 22:21:47.695367 11911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:21:47.696512 11906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:21:47.697389 11906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:21:47.698607 11906 hybrid_clock.cc:648] HybridClock initialized: now 1781907707698562 us; error 79 us; skew 500 ppm
Jun 19 22:21:47 dist-test-slave-k84h krb5kdc[11531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907707, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:21:47.702504 11906 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:21:47.703725 11906 webserver.cc:460] Webserver started at http://127.1.74.131:46815/ using document root <none> and password file <none>
I20260619 22:21:47.704407 11906 fs_manager.cc:359] Metadata directory not provided
I20260619 22:21:47.704522 11906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:21:47.704782 11906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:21:47.707167 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "393912fed16c4999ad5f690521680825"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "a3c2ae248fa9ec23b95032be4d2de674"
server_key_iv: "529563db2640d95525ef4ec868ffe634"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.707793 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "393912fed16c4999ad5f690521680825"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "a3c2ae248fa9ec23b95032be4d2de674"
server_key_iv: "529563db2640d95525ef4ec868ffe634"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.712541 11906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:21:47.715816 11921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.717556 11906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:21:47.717828 11906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "393912fed16c4999ad5f690521680825"
format_stamp: "Formatted at 2026-06-19 22:21:47 on dist-test-slave-k84h"
server_key: "a3c2ae248fa9ec23b95032be4d2de674"
server_key_iv: "529563db2640d95525ef4ec868ffe634"
server_key_version: "encryptionkey@0"
I20260619 22:21:47.717980 11906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:21:47.753335 11906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:21:47.757208 11906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:21:47.757453 11906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:21:47.758155 11906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:21:47.759127 11906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:21:47.759209 11906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.759305 11906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:21:47.759352 11906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:21:47.771533 11906 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:43235
I20260619 22:21:47.771616 12034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:43235 every 8 connection(s)
I20260619 22:21:47.772853 11906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:21:47.783246 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 11906
I20260619 22:21:47.783413 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:21:47.783809 1322 external_mini_cluster.cc:1428] Setting key 89e8840ea583c609937a18946707cc5e
Jun 19 22:21:47 dist-test-slave-k84h krb5kdc[11531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907707, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:21:47.790376 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.775439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:60843 (local address 127.1.74.190:33235)
0619 22:21:47.775646 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.775652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.776757 (+ 1105us) server_negotiation.cc:407] Connection header received
0619 22:21:47.778280 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.778284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.778339 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.778424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:47.780033 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.780809 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.782175 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.782412 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.782521 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.786130 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:47.786135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:47.786137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:47.786238 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:47.788217 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.788854 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.788857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.788859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.788942 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.789327 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.789329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.789330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.789521 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:47.789599 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.790139 (+ 540us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.790248 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 22:21:47.791388 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.775944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33235 (local address 127.1.74.131:60843)
0619 22:21:47.776575 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.776627 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.778022 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.778621 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.778629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.779174 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:21:47.779783 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.779798 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.781015 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.781018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.782015 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.782025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.782615 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.782619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.782798 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.783679 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:21:47.783704 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:21:47.785832 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:21:47.788369 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:47.788377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:47.788397 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:47.788747 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:47.789059 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:21:47.789061 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:21:47.789063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:21:47.789231 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:21:47.789645 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:21:47.789651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:21:47.789918 (+ 267us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.790094 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.790385 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":172,"threads_started":1}
I20260619 22:21:47.792656 12035 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33235
I20260619 22:21:47.792907 12035 heartbeater.cc:464] Registering TS with master...
I20260619 22:21:47.793429 12035 heartbeater.cc:511] Master 127.1.74.190:33235 requested a full tablet report, sending...
I20260619 22:21:47.794749 11576 ts_manager.cc:194] Registered new tserver with Master: 393912fed16c4999ad5f690521680825 (127.1.74.131:43235)
I20260619 22:21:47.795442 11576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:60843
I20260619 22:21:47.798918 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:21:47.811357 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.801891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.1.74.190:33235)
0619 22:21:47.802063 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.802067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.802086 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:21:47.802130 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.802132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.802172 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.802299 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:21:47.803178 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.803728 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.804657 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.804893 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.805011 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.807193 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:21:47.807198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:21:47.807200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:21:47.807239 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:21:47.808977 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.809533 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.809539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.809546 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.809627 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:21:47.810188 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:21:47.810192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:21:47.810194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:21:47.810505 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:21:47.810681 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.811003 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.811172 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 22:21:47.815689 11576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:21:47.819542 11576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:21:47.836364 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.829896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52054 (local address 127.1.74.130:37287)
0619 22:21:47.830734 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.830748 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.830776 (+ 28us) server_negotiation.cc:407] Connection header received
0619 22:21:47.831076 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.831083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.831319 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.831490 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:47.832072 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.833253 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.835000 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.835602 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.835744 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.835910 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.836065 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.836198 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":144,"threads_started":1}
I20260619 22:21:47.836349 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.829857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:44425 (local address 127.0.0.1:59758)
0619 22:21:47.831147 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.831164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.831270 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.831795 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.831798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.831817 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:47.832229 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.832235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.833595 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.833694 (+ 99us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.834681 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.834690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.835992 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.835998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.836095 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.836114 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.836194 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.836249 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"mutex_wait_us":265,"thread_start_us":149,"threads_started":1}
I20260619 22:21:47.836745 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.830231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.1.74.129:44425)
0619 22:21:47.831002 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.831009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.831382 (+ 373us) server_negotiation.cc:407] Connection header received
0619 22:21:47.831473 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.831479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.831655 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.831818 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:47.832368 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.833479 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.834819 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.835690 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.835801 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.835933 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.836434 (+ 501us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.836563 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":62,"threads_started":1}
I20260619 22:21:47.836354 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.829757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:37287 (local address 127.0.0.1:52054)
0619 22:21:47.830649 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.830709 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.830865 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.831702 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.831708 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.831731 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:47.831940 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.831949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.833402 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.833406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.834846 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.834881 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.835749 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.835753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.835865 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.835901 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.836022 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.836165 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260619 22:21:47.838701 12050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.830608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:43235 (local address 127.0.0.1:58078)
0619 22:21:47.831259 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.831272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.831350 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.831943 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.831949 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.831968 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:47.832226 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.832235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.834937 (+ 2702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.834940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.836046 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.836056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.838255 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.838259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.838367 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.838393 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.838479 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.838558 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":10,"thread_start_us":92,"threads_started":1}
I20260619 22:21:47.838783 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.831053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.1.74.131:43235)
0619 22:21:47.831493 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.831504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.831524 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:21:47.831596 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.831601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.831772 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.831970 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:47.833621 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.834787 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.836184 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.838108 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.838232 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.838375 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.838526 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.838636 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":154,"threads_started":1}
I20260619 22:21:47.839995 11835 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5b16aca4ce427cb18621f25d0730e9 (DEFAULT_TABLE table=test-table [id=cef00261f3394fecaa2408de4a3de47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:47.840024 11701 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5b16aca4ce427cb18621f25d0730e9 (DEFAULT_TABLE table=test-table [id=cef00261f3394fecaa2408de4a3de47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:47.841135 11701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5b16aca4ce427cb18621f25d0730e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:47.841137 11835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5b16aca4ce427cb18621f25d0730e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:47.842177 11969 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5b16aca4ce427cb18621f25d0730e9 (DEFAULT_TABLE table=test-table [id=cef00261f3394fecaa2408de4a3de47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:21:47.843401 11969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5b16aca4ce427cb18621f25d0730e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:21:47.848644 12052 tablet_bootstrap.cc:492] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: Bootstrap starting.
I20260619 22:21:47.850287 12053 tablet_bootstrap.cc:492] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: Bootstrap starting.
I20260619 22:21:47.850653 12054 tablet_bootstrap.cc:492] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Bootstrap starting.
I20260619 22:21:47.852120 12052 tablet_bootstrap.cc:654] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:47.852751 12053 tablet_bootstrap.cc:654] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:47.853065 12054 tablet_bootstrap.cc:654] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Neither blocks nor log segments found. Creating new log.
I20260619 22:21:47.853369 12052 log.cc:824] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:47.853482 12053 log.cc:824] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:47.853917 12054 log.cc:824] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Log is configured to *not* fsync() on all Append() calls
I20260619 22:21:47.855681 12053 tablet_bootstrap.cc:492] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: No bootstrap required, opened a new log
I20260619 22:21:47.855880 12052 tablet_bootstrap.cc:492] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: No bootstrap required, opened a new log
I20260619 22:21:47.855959 12053 ts_tablet_manager.cc:1397] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:21:47.856004 12054 tablet_bootstrap.cc:492] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: No bootstrap required, opened a new log
I20260619 22:21:47.856089 12052 ts_tablet_manager.cc:1397] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 22:21:47.856146 12054 ts_tablet_manager.cc:1397] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:21:47.859731 12053 raft_consensus.cc:348] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.860045 12053 raft_consensus.cc:374] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:47.860229 12053 raft_consensus.cc:729] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19e9294cf514a96b4cfc3bc3d1e7227, State: Initialized, Role: FOLLOWER
I20260619 22:21:47.860133 12054 raft_consensus.cc:348] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.860133 12052 raft_consensus.cc:348] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.860383 12052 raft_consensus.cc:374] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:47.860383 12054 raft_consensus.cc:374] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:21:47.860433 12054 raft_consensus.cc:729] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393912fed16c4999ad5f690521680825, State: Initialized, Role: FOLLOWER
I20260619 22:21:47.860442 12052 raft_consensus.cc:729] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0807e48d9b446658783af9697f23643, State: Initialized, Role: FOLLOWER
I20260619 22:21:47.860754 12053 consensus_queue.cc:260] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.860922 12054 consensus_queue.cc:260] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.860921 12052 consensus_queue.cc:260] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.861515 11767 heartbeater.cc:503] Master 127.1.74.190:33235 was elected leader, sending a full tablet report...
I20260619 22:21:47.861914 12053 ts_tablet_manager.cc:1428] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:21:47.861917 12052 ts_tablet_manager.cc:1428] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260619 22:21:47.861917 12054 ts_tablet_manager.cc:1428] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:21:47.862267 12035 heartbeater.cc:503] Master 127.1.74.190:33235 was elected leader, sending a full tablet report...
I20260619 22:21:47.862708 11901 heartbeater.cc:503] Master 127.1.74.190:33235 was elected leader, sending a full tablet report...
I20260619 22:21:47.925413 12058 raft_consensus.cc:482] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:21:47.925727 12058 raft_consensus.cc:504] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.927433 12058 leader_election.cc:288] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0807e48d9b446658783af9697f23643 (127.1.74.130:37287), 393912fed16c4999ad5f690521680825 (127.1.74.131:43235)
I20260619 22:21:47.932475 12048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.927863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:45155 (local address 127.1.74.130:37287)
0619 22:21:47.928004 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.928010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.928291 (+ 281us) server_negotiation.cc:407] Connection header received
0619 22:21:47.928471 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.928474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.928530 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.928623 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:47.929037 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.929808 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.931337 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.931791 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.931898 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.931943 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.932250 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.932338 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:21:47.932565 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.927934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:60799 (local address 127.1.74.131:43235)
0619 22:21:47.928082 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.928090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.928310 (+ 220us) server_negotiation.cc:407] Connection header received
0619 22:21:47.928485 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.928488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.928551 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.928644 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:21:47.928981 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.929809 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.931278 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.931896 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.932054 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.932116 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.932368 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.932445 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 22:21:47.932571 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.927695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:37287 (local address 127.1.74.129:45155)
0619 22:21:47.928170 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.928184 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.928345 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.928643 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.928646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.928663 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:47.928856 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.928862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.929955 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.929959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.931183 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.931197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.931916 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.931922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.932112 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.932146 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.932231 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.932316 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":145,"spinlock_wait_cycles":11520,"thread_start_us":145,"threads_started":1}
I20260619 22:21:47.932588 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.927820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:43235 (local address 127.1.74.129:60799)
0619 22:21:47.928205 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:47.928217 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:21:47.928369 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:21:47.928654 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:21:47.928657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:21:47.928673 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:21:47.928847 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.928853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.929951 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.929956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.931171 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:21:47.931182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.932041 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.932044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:21:47.932214 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.932240 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:21:47.932338 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 22:21:47.932417 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":29,"spinlock_wait_cycles":1920,"thread_start_us":121,"threads_started":1}
I20260619 22:21:47.933396 11855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5b16aca4ce427cb18621f25d0730e9" candidate_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0807e48d9b446658783af9697f23643" is_pre_election: true
I20260619 22:21:47.933403 11989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5b16aca4ce427cb18621f25d0730e9" candidate_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393912fed16c4999ad5f690521680825" is_pre_election: true
I20260619 22:21:47.933785 11855 raft_consensus.cc:2454] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b19e9294cf514a96b4cfc3bc3d1e7227 in term 0.
I20260619 22:21:47.933787 11989 raft_consensus.cc:2454] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b19e9294cf514a96b4cfc3bc3d1e7227 in term 0.
I20260619 22:21:47.934381 11654 leader_election.cc:302] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 393912fed16c4999ad5f690521680825, a0807e48d9b446658783af9697f23643, b19e9294cf514a96b4cfc3bc3d1e7227; no voters:
I20260619 22:21:47.934755 12058 raft_consensus.cc:2781] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:21:47.934864 12058 raft_consensus.cc:482] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:21:47.934948 12058 raft_consensus.cc:3037] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:47.936180 12058 raft_consensus.cc:504] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.936656 12058 leader_election.cc:288] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [CANDIDATE]: Term 1 election: Requested vote from peers a0807e48d9b446658783af9697f23643 (127.1.74.130:37287), 393912fed16c4999ad5f690521680825 (127.1.74.131:43235)
I20260619 22:21:47.937152 11989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5b16aca4ce427cb18621f25d0730e9" candidate_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393912fed16c4999ad5f690521680825"
I20260619 22:21:47.937152 11855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5b16aca4ce427cb18621f25d0730e9" candidate_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0807e48d9b446658783af9697f23643"
I20260619 22:21:47.937309 11855 raft_consensus.cc:3037] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:47.937309 11989 raft_consensus.cc:3037] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:21:47.938757 11989 raft_consensus.cc:2454] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b19e9294cf514a96b4cfc3bc3d1e7227 in term 1.
I20260619 22:21:47.938757 11855 raft_consensus.cc:2454] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b19e9294cf514a96b4cfc3bc3d1e7227 in term 1.
I20260619 22:21:47.939177 11657 leader_election.cc:302] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 393912fed16c4999ad5f690521680825, a0807e48d9b446658783af9697f23643, b19e9294cf514a96b4cfc3bc3d1e7227; no voters:
I20260619 22:21:47.939396 12058 raft_consensus.cc:2781] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:21:47.939675 12058 raft_consensus.cc:686] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 1 LEADER]: Becoming Leader. State: Replica: b19e9294cf514a96b4cfc3bc3d1e7227, State: Running, Role: LEADER
I20260619 22:21:47.940078 12058 consensus_queue.cc:237] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } }
I20260619 22:21:47.944136 11576 catalog_manager.cc:5495] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 reported cstate change: term changed from 0 to 1, leader changed from <none> to b19e9294cf514a96b4cfc3bc3d1e7227 (127.1.74.129). New cstate: current_term: 1 leader_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b19e9294cf514a96b4cfc3bc3d1e7227" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 } health_report { overall_health: UNKNOWN } } }
I20260619 22:21:47.965170 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:47.960430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.1.74.129:44425)
0619 22:21:47.960581 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 22:21:47.960587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:21:47.960603 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:21:47.960694 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:21:47.960697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:21:47.960769 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:21:47.960892 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:21:47.961136 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.961925 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.963037 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:21:47.963350 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:21:47.963712 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:21:47.964308 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:21:47.964748 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:21:47.964899 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 22:21:47.964945 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 22:21:47.965012 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:21:47.971947 11989 raft_consensus.cc:1261] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 1 FOLLOWER]: Refusing update from remote peer b19e9294cf514a96b4cfc3bc3d1e7227: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:47.971946 11855 raft_consensus.cc:1261] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 1 FOLLOWER]: Refusing update from remote peer b19e9294cf514a96b4cfc3bc3d1e7227: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:21:47.972744 12058 consensus_queue.cc:1035] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393912fed16c4999ad5f690521680825" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 43235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:47.973001 12063 consensus_queue.cc:1035] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0807e48d9b446658783af9697f23643" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 37287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:21:47.980968 12067 mvcc.cc:205] Tried to move back new op lower bound from 7298693971844747264 to 7298693971725623296. Current Snapshot: MvccSnapshot[applied={T|T < 7298693971844747264}]
I20260619 22:21:47.990348 11575 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:47.990497 11575 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:21:47.992892 11575 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ae2f682f2c84c04ba9d77da9faa19f6: Sending DeleteTablet for 3 replicas of tablet 5b5b16aca4ce427cb18621f25d0730e9
I20260619 22:21:47.993665 11835 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5b16aca4ce427cb18621f25d0730e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:47 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:52054
I20260619 22:21:47.993691 11701 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5b16aca4ce427cb18621f25d0730e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:47 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:59758
I20260619 22:21:47.994113 11969 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5b16aca4ce427cb18621f25d0730e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:21:47 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58078
I20260619 22:21:47.994144 12077 tablet_replica.cc:330] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: stopping tablet replica
I20260619 22:21:47.994462 12077 raft_consensus.cc:2229] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:47.994589 12078 tablet_replica.cc:330] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: stopping tablet replica
I20260619 22:21:47.994766 12077 raft_consensus.cc:2258] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:47.994961 12078 raft_consensus.cc:2229] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:21:47.995163 12076 tablet_replica.cc:330] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227: stopping tablet replica
I20260619 22:21:47.995404 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11638
I20260619 22:21:47.995407 12076 raft_consensus.cc:2229] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:21:47.995755 12076 raft_consensus.cc:2258] T 5b5b16aca4ce427cb18621f25d0730e9 P b19e9294cf514a96b4cfc3bc3d1e7227 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:47.995420 12078 raft_consensus.cc:2258] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:21:47.997556 12078 ts_tablet_manager.cc:1905] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:47.999852 12078 ts_tablet_manager.cc:1918] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:21:47.999927 12078 log.cc:1196] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/5b5b16aca4ce427cb18621f25d0730e9
I20260619 22:21:48.000250 12078 ts_tablet_manager.cc:1939] T 5b5b16aca4ce427cb18621f25d0730e9 P 393912fed16c4999ad5f690521680825: Deleting consensus metadata
I20260619 22:21:48.001266 11563 catalog_manager.cc:4841] TS 393912fed16c4999ad5f690521680825 (127.1.74.131:43235): tablet 5b5b16aca4ce427cb18621f25d0730e9 (table test-table [id=cef00261f3394fecaa2408de4a3de47a]) successfully deleted
W20260619 22:21:48.002280 11562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:44425 (error 108)
I20260619 22:21:48.002302 12077 ts_tablet_manager.cc:1905] T 5b5b16aca4ce427cb18621f25d0730e9 P a0807e48d9b446658783af9697f23643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:21:48.003149 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11772
I20260619 22:21:48.003366 12050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:48.003046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:44425 (local address 127.0.0.1:59778)
0619 22:21:48.003190 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:48.003267 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:44425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 22:21:48.004789 11562 catalog_manager.cc:4584] TS b19e9294cf514a96b4cfc3bc3d1e7227 (127.1.74.129:44425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b5b16aca4ce427cb18621f25d0730e9: Network error: Client connection negotiation failed: client connection to 127.1.74.129:44425: connect: Connection refused (error 111)
I20260619 22:21:48.008963 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11906
I20260619 22:21:48.010674 12050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:21:48.009175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:37287 (local address 127.0.0.1:52056)
0619 22:21:48.010558 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0619 22:21:48.010608 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:37287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1281}
I20260619 22:21:48.014164 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 11544
2026-06-19T22:21:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2481 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:21:48 dist-test-slave-k84h krb5kdc[12082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:21:48 dist-test-slave-k84h krb5kdc[12082](info): set up 2 sockets
Jun 19 22:21:48 dist-test-slave-k84h krb5kdc[12082](info): commencing operation
krb5kdc: starting...
W20260619 22:21:49.096264 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:21:49 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:21:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:21:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/rangeradmin_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/rangeradmin_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/rangerlookup_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/rangerlookup_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
I20260619 22:21:49.282836 1322 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSWejqq/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/postgres -l logfile start
2026-06-19 22:21:52.028 UTC [12112] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 22:21:52.028 UTC [12112] LOG: listening on IPv4 address "127.1.74.148", port 35509
2026-06-19 22:21:52.032 UTC [12112] LOG: listening on Unix socket "/tmp/.s.PGSQL.35509"
2026-06-19 22:21:52.054 UTC [12115] LOG: database system was shut down at 2026-06-19 22:21:50 UTC
2026-06-19 22:21:52.058 UTC [12112] LOG: database system is ready to accept connections
I20260619 22:21:53.023777 1322 mini_postgres.cc:96] Postgres bound to 35509
2026-06-19 22:21:53.030 UTC [12123] FATAL: database "slave" does not exist
127.1.74.148:35509 - accepting connections
I20260619 22:21:53.031062 1322 mini_ranger.cc:161] Starting Ranger...
I20260619 22:21:53.046355 1322 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 22:21:53.562916 1322 mini_ranger.cc:87] Created ranger Postgres database
I20260619 22:21:53.563045 1322 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-admin
2026-06-19 22:21:53,846 [I] DB FLAVOR :POSTGRES
2026-06-19 22:21:53,846 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 22:21:53,846 [I] Checking connection..
2026-06-19 22:21:53,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 22:21:54,115 [I] Checking connection passed.
2026-06-19 22:21:54,115 [I] --------- Verifying version history table ---------
2026-06-19 22:21:54,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 22:21:54,367 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 22:21:54,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 22:21:54,608 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 22:21:54,608 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 22:21:54,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 22:21:54.852 UTC [12187] WARNING: there is no transaction in progress
2026-06-19 22:21:54,859 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 22:21:54,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 22:21:55,107 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 22:21:55,107 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 22:21:55,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 22:21:55,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 22:21:55,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-k84h.c.gcp-upstream.internal','N') ;"
2026-06-19 22:21:55,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 22:21:56,098 [I] Table x_portal_user does not exist in database ranger
2026-06-19 22:21:56,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 22:21:56,351 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 22:21:56,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 22:21:56,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 22:21:56,822 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 22:21:56,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 22:21:57.868 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:57.886 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:57.903 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:57.919 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:57.937 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:57.955 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.008 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.015 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.029 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.039 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.049 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.062 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.071 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.077 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.084 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58.091 UTC [12295] WARNING: there is no transaction in progress
2026-06-19 22:21:58,755 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 22:21:58,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 22:21:58,979 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 22:21:58,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 22:21:59,222 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 22:21:59,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 22:21:59,506 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 22:21:59,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 22:21:59,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 22:21:59,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-k84h.c.gcp-upstream.internal';"
2026-06-19 22:22:00,218 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 22:22:00,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 22:22:00,477 [I] DB_PATCHES have already been applied
I20260619 22:22:00.483131 1322 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSWejqq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 22:22:00.483223 1322 mini_ranger.cc:193] Using host: 127.1.74.148
I20260619 22:22:00.486783 1322 mini_ranger.cc:227] Ranger admin URL: http://127.1.74.148:46851
Jun 19, 2026 10:22:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:22:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:22:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 10:22:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 10:22:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 10:22:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 10:22:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 22:22:01.625982 1322 mini_ranger.cc:160] Time spent starting Ranger: real 8.595s user 0.004s sys 0.007s
Jun 19, 2026 10:22:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 10:22:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 10:22:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 10:22:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 10:22:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 22:22:22.859735 1322 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/rangerkms_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/rangerkms_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.74.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.74.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 22:22:22 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907742, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 22:22:22.948310 1322 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 22:22:22.957312 1322 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 22:22:23.937271 1322 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 22:22:23.937389 1322 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms
I20260619 22:22:23.937497 1322 mini_ranger_kms.cc:227] Using postgres at 127.1.74.148:35509
2026-06-19 22:22:24,129 [I] DB FLAVOR :POSTGRES
2026-06-19 22:22:24,129 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 22:22:24,129 [I] Checking connection
2026-06-19 22:22:24,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 22:22:24,400 [I] connection success
2026-06-19 22:22:24,400 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 22:22:24,400 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 22:22:24,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 22:22:24,646 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 22:22:24,646 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 22:22:24,647 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 22:22:24,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:35509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 22:22:24,930 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 22:22:25.417799 1322 mini_ranger_kms.cc:310] Created kms service
I20260619 22:22:25.644342 1322 mini_ranger_kms.cc:326] Created kudu user
I20260619 22:22:25.690472 1322 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 22:22:25 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907742, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:22:26.110953 1322 mini_ranger_kms.cc:384] Added ranger policy
I20260619 22:22:26.111112 1322 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 22:22:26.111133 1322 mini_ranger_kms.cc:240] Using host: 127.1.74.148
I20260619 22:22:26.115340 1322 mini_ranger_kms.cc:276] Ranger KMS PID: 12475
I20260619 22:22:26.115454 1322 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.74.148:41405
Jun 19, 2026 10:22:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:22:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:22:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 10:22:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 10:22:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 10:22:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 22:22:28 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781907748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.74.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:22:28.383554 1322 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.435s user 0.010s sys 0.165s
I20260619 22:22:28.383693 1322 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 22:22:28.383810 1322 mini_ranger_kms.cc:401] 127.1.74.148:41405/kms/v1/keys
Jun 19 22:22:28 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.74.148@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
Jun 19 22:22:29 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.74.148@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
Jun 19 22:22:31 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:22:31.343142 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:36791
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:36791
--ranger_config_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:41405/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:22:31.457779 12526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:22:31.458057 12526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:22:31.458097 12526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:22:31.462319 12526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:22:31.462399 12526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:22:31.462419 12526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:22:31.462435 12526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:22:31.462450 12526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:22:31.469316 12526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.74.148:41405/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:36791
--ranger_config_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:36791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:22:31.470494 12526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:22:31.471638 12526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:22:31.481613 12531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:22:31.481779 12526 server_base.cc:1034] running on GCE node
W20260619 22:22:31.481596 12532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:22:31.482025 12534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:22:31.482617 12526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:22:31.483872 12526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:22:31.485054 12526 hybrid_clock.cc:648] HybridClock initialized: now 1781907751485038 us; error 39 us; skew 500 ppm
Jun 19 22:22:31 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907751, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:22:31.493142 12526 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:22:31.494284 12526 webserver.cc:460] Webserver started at http://127.1.74.190:38231/ using document root <none> and password file <none>
I20260619 22:22:31.494930 12526 fs_manager.cc:359] Metadata directory not provided
I20260619 22:22:31.494999 12526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:22:31.495224 12526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:22:31 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907751, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:22:31.551812 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "fa7cf44b7ba14ab6926452b1678f433b"
format_stamp: "Formatted at 2026-06-19 22:22:31 on dist-test-slave-k84h"
server_key: "8ff95fdf49ea9b18fbc18eb9ed2debbf"
server_key_iv: "5f619670d8731ca8908bf77f34ad48c2"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:31.552646 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "fa7cf44b7ba14ab6926452b1678f433b"
format_stamp: "Formatted at 2026-06-19 22:22:31 on dist-test-slave-k84h"
server_key: "8ff95fdf49ea9b18fbc18eb9ed2debbf"
server_key_iv: "5f619670d8731ca8908bf77f34ad48c2"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:31.557132 12526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:22:31.576049 12541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:31.577762 12526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:22:31.577929 12526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "fa7cf44b7ba14ab6926452b1678f433b"
format_stamp: "Formatted at 2026-06-19 22:22:31 on dist-test-slave-k84h"
server_key: "8ff95fdf49ea9b18fbc18eb9ed2debbf"
server_key_iv: "5f619670d8731ca8908bf77f34ad48c2"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:31.578089 12526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:22:31.607028 12526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:22:31.610232 12526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:22:31.610473 12526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:22:31.618497 12526 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:36791
I20260619 22:22:31.618526 12593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:36791 every 8 connection(s)
I20260619 22:22:31.619585 12526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:22:31.621449 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 12526
I20260619 22:22:31.621595 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:22:31.622002 1322 external_mini_cluster.cc:1428] Setting key a5d375f563c0b132d1eba493c707c195
I20260619 22:22:31.623224 12594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:22:31.630355 12594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b: Bootstrap starting.
Jun 19 22:22:31 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:22:31.633267 12594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b: Neither blocks nor log segments found. Creating new log.
I20260619 22:22:31.634734 12594 log.cc:824] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b: Log is configured to *not* fsync() on all Append() calls
I20260619 22:22:31.637307 12594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b: No bootstrap required, opened a new log
I20260619 22:22:31.638561 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:31.623577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.1.74.190:36791)
0619 22:22:31.624009 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:31.624029 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:31.624071 (+ 42us) server_negotiation.cc:407] Connection header received
0619 22:22:31.624952 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:31.624975 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:31.625405 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:31.625805 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:22:31.626525 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:31.628075 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:31.628991 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:31.629700 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:31.629862 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:31.632919 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:22:31.632926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:22:31.632943 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:22:31.632982 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:22:31.635060 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:31.635632 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:31.635665 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:31.635672 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:31.635806 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:31.636112 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:31.636115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:31.636116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:31.636637 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:22:31.636828 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:31.637125 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 22:22:31.637598 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":128,"threads_started":1}
I20260619 22:22:31.641048 12594 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7cf44b7ba14ab6926452b1678f433b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36791 } }
I20260619 22:22:31.641268 12594 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:22:31.641302 12594 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7cf44b7ba14ab6926452b1678f433b, State: Initialized, Role: FOLLOWER
I20260619 22:22:31.641654 12594 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7cf44b7ba14ab6926452b1678f433b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36791 } }
I20260619 22:22:31.641762 12594 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:22:31.641799 12594 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:22:31.641860 12594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:22:31.643286 12594 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7cf44b7ba14ab6926452b1678f433b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36791 } }
I20260619 22:22:31.643625 12594 leader_election.cc:302] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa7cf44b7ba14ab6926452b1678f433b; no voters:
I20260619 22:22:31.643906 12594 leader_election.cc:288] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:22:31.644120 12599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:22:31.644500 12599 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [term 1 LEADER]: Becoming Leader. State: Replica: fa7cf44b7ba14ab6926452b1678f433b, State: Running, Role: LEADER
I20260619 22:22:31.644824 12599 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7cf44b7ba14ab6926452b1678f433b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36791 } }
I20260619 22:22:31.645555 12594 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:22:31.646766 12600 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa7cf44b7ba14ab6926452b1678f433b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7cf44b7ba14ab6926452b1678f433b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36791 } } }
I20260619 22:22:31.646775 12601 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa7cf44b7ba14ab6926452b1678f433b. Latest consensus state: current_term: 1 leader_uuid: "fa7cf44b7ba14ab6926452b1678f433b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7cf44b7ba14ab6926452b1678f433b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36791 } } }
I20260619 22:22:31.646992 12601 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [sys.catalog]: This master's current role is: LEADER
I20260619 22:22:31.647313 12600 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b [sys.catalog]: This master's current role is: LEADER
I20260619 22:22:31.647420 12608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:22:31.650029 12608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:22:31.650490 12594 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 22:22:31.659948 12608 catalog_manager.cc:1347] Generated new cluster ID: 577a259878c04123a1976598b0f33dc1
I20260619 22:22:31.660099 12608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:22:31.676656 12608 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:22:31.678131 12608 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:22:31.691303 12608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b: Generated new TSK 0
I20260619 22:22:31.692104 12608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:22:32.398813 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:41405/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:36791
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:22:32.566784 12639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:22:32.567227 12639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:22:32.567302 12639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:22:32.574307 12639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:22:32.574416 12639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:22:32.574512 12639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:22:32.583130 12639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.74.148:41405/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:22:32.584769 12639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:22:32.586179 12639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:22:32.598483 12639 server_base.cc:1034] running on GCE node
W20260619 22:22:32.598690 12646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:22:32.598977 12648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:22:32.599740 12645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:22:32.600302 12639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:22:32.601125 12639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:22:32.602330 12639 hybrid_clock.cc:648] HybridClock initialized: now 1781907752602297 us; error 46 us; skew 500 ppm
Jun 19 22:22:32 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:22:32.605334 12639 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:22:32.606366 12639 webserver.cc:460] Webserver started at http://127.1.74.129:39675/ using document root <none> and password file <none>
I20260619 22:22:32.607026 12639 fs_manager.cc:359] Metadata directory not provided
I20260619 22:22:32.607121 12639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:22:32.607334 12639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:22:32 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:22:32.633277 12639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "cf05885e75f04ff0b4d5563cc5cd09e7"
format_stamp: "Formatted at 2026-06-19 22:22:32 on dist-test-slave-k84h"
server_key: "dcffef9d809a8a1f236d9fec1eb0703a"
server_key_iv: "5b3503c6f68bbe1b09144865f5310ff4"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:32.634177 12639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "cf05885e75f04ff0b4d5563cc5cd09e7"
format_stamp: "Formatted at 2026-06-19 22:22:32 on dist-test-slave-k84h"
server_key: "dcffef9d809a8a1f236d9fec1eb0703a"
server_key_iv: "5b3503c6f68bbe1b09144865f5310ff4"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:32.639552 12639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 22:22:32.661791 12656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:32.663671 12639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:22:32.663810 12639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "cf05885e75f04ff0b4d5563cc5cd09e7"
format_stamp: "Formatted at 2026-06-19 22:22:32 on dist-test-slave-k84h"
server_key: "dcffef9d809a8a1f236d9fec1eb0703a"
server_key_iv: "5b3503c6f68bbe1b09144865f5310ff4"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:32.663945 12639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:22:32.694482 12639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:22:32.698650 12639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:22:32.698948 12639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:22:32.699756 12639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:22:32.700970 12639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:22:32.701081 12639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:32.701218 12639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:22:32.701308 12639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:32.716754 12639 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:44291
I20260619 22:22:32.718644 12639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:22:32.719714 12769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:44291 every 8 connection(s)
I20260619 22:22:32.731123 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 12639
I20260619 22:22:32.731253 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:22:32.731575 1322 external_mini_cluster.cc:1428] Setting key f6d5c5b7aab0a0350947b5c6349a5a10
Jun 19 22:22:32 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:22:32.739300 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:32.721728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:39633 (local address 127.1.74.190:36791)
0619 22:22:32.724007 (+ 2279us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:32.724012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:32.724031 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:22:32.724405 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:32.724409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:32.724469 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:32.724582 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:22:32.726173 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:32.727176 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:32.729024 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:32.729403 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:32.729546 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:32.732475 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:22:32.732480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:22:32.732484 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:22:32.732514 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:22:32.734882 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:32.735651 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:32.735655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:32.735657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:32.735763 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:32.736239 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:32.736242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:32.736244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:32.736512 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:22:32.736622 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:32.738972 (+ 2350us) server_negotiation.cc:299] Negotiation successful
0619 22:22:32.739117 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2189,"thread_start_us":88,"threads_started":1}
I20260619 22:22:32.739653 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:32.722243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:36791 (local address 127.1.74.129:39633)
0619 22:22:32.722923 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:32.722985 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:32.724126 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:32.724792 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:32.724801 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:32.725277 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:22:32.725983 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:32.726003 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:32.727334 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:32.727338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:32.728868 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:32.728881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:32.729591 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:32.729597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:32.729785 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:32.730474 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:22:32.730492 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:22:32.732228 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:22:32.735042 (+ 2814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:22:32.735050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:22:32.735071 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:22:32.735513 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:22:32.735883 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:22:32.735887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:22:32.735889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:22:32.736108 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:22:32.736686 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:22:32.736694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:22:32.737080 (+ 386us) client_negotiation.cc:769] Sending connection context
0619 22:22:32.737457 (+ 377us) client_negotiation.cc:240] Negotiation successful
0619 22:22:32.737845 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":164,"threads_started":1}
I20260619 22:22:32.741242 12771 heartbeater.cc:347] Connected to a master server at 127.1.74.190:36791
I20260619 22:22:32.741516 12771 heartbeater.cc:464] Registering TS with master...
I20260619 22:22:32.742063 12771 heartbeater.cc:511] Master 127.1.74.190:36791 requested a full tablet report, sending...
I20260619 22:22:32.743880 12556 ts_manager.cc:194] Registered new tserver with Master: cf05885e75f04ff0b4d5563cc5cd09e7 (127.1.74.129:44291)
I20260619 22:22:32.745386 12556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:39633
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:22:32.787142 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:41405/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:36791
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:22:32 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 22:22:32.949653 12778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:22:32.950070 12778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:22:32.950139 12778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:22:32.956429 12778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:22:32.956586 12778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:22:32.956712 12778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:22:32.964946 12778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.74.148:41405/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:22:32.966423 12778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:22:32.967828 12778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:22:32.980062 12785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:22:32.980201 12784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:22:32.980067 12787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:22:32.980844 12778 server_base.cc:1034] running on GCE node
I20260619 22:22:32.981344 12778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:22:32.982143 12778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:22:32.983321 12778 hybrid_clock.cc:648] HybridClock initialized: now 1781907752983298 us; error 39 us; skew 500 ppm
Jun 19 22:22:32 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:22:32.987142 12778 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:22:32.988380 12778 webserver.cc:460] Webserver started at http://127.1.74.130:34775/ using document root <none> and password file <none>
I20260619 22:22:32.989131 12778 fs_manager.cc:359] Metadata directory not provided
I20260619 22:22:32.989229 12778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:22:32.989504 12778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:22:32 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:22:33.005013 12778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "e6334e50a9934772b37e38ad303cd784"
format_stamp: "Formatted at 2026-06-19 22:22:32 on dist-test-slave-k84h"
server_key: "d7c478f4637ee07e5d88165f1f905952"
server_key_iv: "fb6198fa0e2e437ae822e543b7a4df1f"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:33.005869 12778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "e6334e50a9934772b37e38ad303cd784"
format_stamp: "Formatted at 2026-06-19 22:22:32 on dist-test-slave-k84h"
server_key: "d7c478f4637ee07e5d88165f1f905952"
server_key_iv: "fb6198fa0e2e437ae822e543b7a4df1f"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:33.011219 12778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 22:22:33.036463 12794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:33.038422 12778 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260619 22:22:33.038571 12778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "e6334e50a9934772b37e38ad303cd784"
format_stamp: "Formatted at 2026-06-19 22:22:32 on dist-test-slave-k84h"
server_key: "d7c478f4637ee07e5d88165f1f905952"
server_key_iv: "fb6198fa0e2e437ae822e543b7a4df1f"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:33.038719 12778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:22:33.068300 12778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:22:33.072540 12778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:22:33.072779 12778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:22:33.073510 12778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:22:33.075125 12778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:22:33.075205 12778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:33.075266 12778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:22:33.075299 12778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:33.090781 12778 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:40825
I20260619 22:22:33.090834 12907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:40825 every 8 connection(s)
I20260619 22:22:33.092545 12778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:22:33.100250 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 12778
I20260619 22:22:33.100392 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:22:33.100730 1322 external_mini_cluster.cc:1428] Setting key fdee52de4954ca5477a23c7535ba7378
Jun 19 22:22:33 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:22:33.124620 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.095598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:41059 (local address 127.1.74.190:36791)
0619 22:22:33.095784 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.095790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.107595 (+ 11805us) server_negotiation.cc:407] Connection header received
0619 22:22:33.109028 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.109032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.109095 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.109150 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:22:33.111001 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.112007 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.113848 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.114205 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.114287 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.117702 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:22:33.117708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:22:33.117711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:22:33.117742 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:22:33.119716 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:33.120654 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:33.120659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:33.120661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:33.120757 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:33.123077 (+ 2320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:33.123081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:33.123083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:33.123365 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:22:33.123461 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.124264 (+ 803us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.124443 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 22:22:33.126672 12910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.096158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:36791 (local address 127.1.74.130:41059)
0619 22:22:33.107355 (+ 11197us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.107453 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.108716 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.109653 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.109664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.110061 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:22:33.110781 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.110801 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.112198 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.112201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.113685 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.113698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.114538 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.114541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.114780 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.115612 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:22:33.115636 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:22:33.117453 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:22:33.119877 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:22:33.119884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:22:33.119906 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:22:33.120521 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:22:33.120869 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:22:33.120872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:22:33.120874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:22:33.121071 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:22:33.123562 (+ 2491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:22:33.123570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:22:33.123955 (+ 385us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.125079 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.125481 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10849,"thread_start_us":200,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:22:33.133538 12908 heartbeater.cc:347] Connected to a master server at 127.1.74.190:36791
I20260619 22:22:33.133810 12908 heartbeater.cc:464] Registering TS with master...
I20260619 22:22:33.134447 12908 heartbeater.cc:511] Master 127.1.74.190:36791 requested a full tablet report, sending...
I20260619 22:22:33.135854 12556 ts_manager.cc:194] Registered new tserver with Master: e6334e50a9934772b37e38ad303cd784 (127.1.74.130:40825)
I20260619 22:22:33.136678 12556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:41059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:22:33.161600 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:41405/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:36791
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:22:33 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
W20260619 22:22:33.325420 12913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:22:33.325831 12913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:22:33.326045 12913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:22:33.332885 12913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:22:33.333056 12913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:22:33.333213 12913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:22:33.342132 12913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.74.148:41405/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:36791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:22:33.344043 12913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:22:33.345795 12913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:22:33.362535 12919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:22:33.362545 12918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:22:33.363046 12913 server_base.cc:1034] running on GCE node
W20260619 22:22:33.369305 12921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:22:33.370072 12913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:22:33.371237 12913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:22:33.372558 12913 hybrid_clock.cc:648] HybridClock initialized: now 1781907753372490 us; error 93 us; skew 500 ppm
Jun 19 22:22:33 dist-test-slave-k84h krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:22:33.377063 12913 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:22:33.378310 12913 webserver.cc:460] Webserver started at http://127.1.74.131:35479/ using document root <none> and password file <none>
I20260619 22:22:33.379195 12913 fs_manager.cc:359] Metadata directory not provided
I20260619 22:22:33.379349 12913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:22:33.379654 12913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:22:33 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:22:33.420043 12913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "66b62b04306540789c2adceb0d4c2664"
format_stamp: "Formatted at 2026-06-19 22:22:33 on dist-test-slave-k84h"
server_key: "a2897b662efca9ce74572a5e36c41487"
server_key_iv: "b047808a0fb209614ed042f51204c362"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:33.422109 12913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "66b62b04306540789c2adceb0d4c2664"
format_stamp: "Formatted at 2026-06-19 22:22:33 on dist-test-slave-k84h"
server_key: "a2897b662efca9ce74572a5e36c41487"
server_key_iv: "b047808a0fb209614ed042f51204c362"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:33.437539 12913 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.004s sys 0.000s
I20260619 22:22:33.465334 12928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:33.467296 12913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:22:33.467449 12913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "66b62b04306540789c2adceb0d4c2664"
format_stamp: "Formatted at 2026-06-19 22:22:33 on dist-test-slave-k84h"
server_key: "a2897b662efca9ce74572a5e36c41487"
server_key_iv: "b047808a0fb209614ed042f51204c362"
server_key_version: "kuduclusterkey@0"
I20260619 22:22:33.467615 12913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:22:33.500444 12913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:22:33.505103 12913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:22:33.505404 12913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:22:33.506254 12913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:22:33.507483 12913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:22:33.507604 12913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:33.507723 12913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:22:33.507795 12913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:22:33.522768 12913 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:45731
I20260619 22:22:33.524271 12913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:22:33.527300 13041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:45731 every 8 connection(s)
I20260619 22:22:33.532377 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 12913
I20260619 22:22:33.532531 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:22:33.532922 1322 external_mini_cluster.cc:1428] Setting key 88a3514c04d683e45e7d00741cee3ead
Jun 19 22:22:33 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907753, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:22:33.548456 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.531236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:60921 (local address 127.1.74.190:36791)
0619 22:22:33.531386 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.531392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.532484 (+ 1092us) server_negotiation.cc:407] Connection header received
0619 22:22:33.533848 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.533852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.533927 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.533984 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:22:33.536365 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.537327 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.539311 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.539678 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.539814 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.543048 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:22:33.543053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:22:33.543056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:22:33.543089 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:22:33.545038 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:33.546300 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:33.546305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:33.546307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:33.546409 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:33.546915 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:33.546919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:33.546921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:33.547201 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:22:33.547294 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.548129 (+ 835us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.548280 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 22:22:33.549429 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.529909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:36791 (local address 127.1.74.131:60921)
0619 22:22:33.532306 (+ 2397us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.532367 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.533571 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.534442 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.534452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.535380 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:22:33.536175 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.536196 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.537511 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.537515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.539152 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.539167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.539889 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.539893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.540126 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.540928 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:22:33.540953 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:22:33.542784 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:22:33.545179 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:22:33.545186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:22:33.545206 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:22:33.546166 (+ 960us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:22:33.546534 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:22:33.546538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:22:33.546540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:22:33.546742 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:22:33.547419 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:22:33.547426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:22:33.547791 (+ 365us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.548015 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.548359 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2159,"thread_start_us":165,"threads_started":1}
I20260619 22:22:33.551191 13042 heartbeater.cc:347] Connected to a master server at 127.1.74.190:36791
I20260619 22:22:33.551431 13042 heartbeater.cc:464] Registering TS with master...
I20260619 22:22:33.551990 13042 heartbeater.cc:511] Master 127.1.74.190:36791 requested a full tablet report, sending...
I20260619 22:22:33.553164 12556 ts_manager.cc:194] Registered new tserver with Master: 66b62b04306540789c2adceb0d4c2664 (127.1.74.131:45731)
I20260619 22:22:33.554013 12556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:60921
I20260619 22:22:33.559253 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:22:33.572613 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.561799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.1.74.190:36791)
0619 22:22:33.561940 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.561945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.562190 (+ 245us) server_negotiation.cc:407] Connection header received
0619 22:22:33.562377 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.562380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.562431 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.562485 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:22:33.563624 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.564530 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.565870 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.566247 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.566337 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.568487 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:22:33.568492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:22:33.568494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:22:33.568524 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:22:33.570371 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:33.571037 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:33.571041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:33.571043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:33.571131 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:22:33.571477 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:22:33.571481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:22:33.571483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:22:33.571726 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:22:33.571849 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.572287 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.572429 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:22:33.577209 12556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:22:33.582007 12556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:22:33.610419 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.598210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:45731 (local address 127.0.0.1:59688)
0619 22:22:33.600686 (+ 2476us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.600749 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.600892 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.604200 (+ 3308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.604207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.604235 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:22:33.604416 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.604424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.606027 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.606031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.607440 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.607454 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.609706 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.609713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.609915 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.609986 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.610095 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.610239 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1954,"thread_start_us":120,"threads_started":1}
I20260619 22:22:33.611738 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.599104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.1.74.131:45731)
0619 22:22:33.603112 (+ 4008us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.603124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.603150 (+ 26us) server_negotiation.cc:407] Connection header received
0619 22:22:33.603229 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.603235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.603457 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.604591 (+ 1134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:22:33.604751 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.605878 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.607648 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.608531 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.611196 (+ 2665us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.611384 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.611446 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.611558 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3893,"thread_start_us":128,"threads_started":1}
I20260619 22:22:33.615278 12975 tablet_service.cc:1467] Processing CreateTablet for tablet 55171a9ea959456b8a4b311a8e2441b4 (DEFAULT_TABLE table=test-table [id=4383c5a390d742ccb81184e596b01954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:22:33.616510 12975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55171a9ea959456b8a4b311a8e2441b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:22:33.622344 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.599939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:44291 (local address 127.0.0.1:54934)
0619 22:22:33.601255 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.601269 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.601726 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.617516 (+ 15790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.617519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.617542 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:22:33.617653 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.617660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.619266 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.619269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.620539 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.620550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.621916 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.621919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.622021 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.622046 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.622114 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.622198 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"thread_start_us":103,"threads_started":1}
I20260619 22:22:33.622828 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.600099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.1.74.129:44291)
0619 22:22:33.601853 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.601862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.601888 (+ 26us) server_negotiation.cc:407] Connection header received
0619 22:22:33.616865 (+ 14977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.616874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.617123 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.617271 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:22:33.617811 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.619113 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.620688 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.621582 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.621666 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.621816 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.622571 (+ 755us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.622684 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1645,"thread_start_us":126,"threads_started":1}
I20260619 22:22:33.625038 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.600995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:40825 (local address 127.0.0.1:53646)
0619 22:22:33.601501 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.601516 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.601603 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.609437 (+ 7834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.609441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.609466 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:22:33.609575 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.609582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.618181 (+ 8599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.618185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.619534 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.619543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.624623 (+ 5080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.624628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.624730 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.624759 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.624813 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.624894 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":92,"threads_started":1}
I20260619 22:22:33.625849 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.601427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.1.74.130:40825)
0619 22:22:33.608849 (+ 7422us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.608879 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.608909 (+ 30us) server_negotiation.cc:407] Connection header received
0619 22:22:33.608994 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.609001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.609244 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.609444 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:22:33.609739 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.618029 (+ 8290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.623543 (+ 5514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.624459 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.625351 (+ 892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.625538 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.625598 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.625695 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7278,"thread_start_us":97,"threads_started":1}
I20260619 22:22:33.626471 12702 tablet_service.cc:1467] Processing CreateTablet for tablet 55171a9ea959456b8a4b311a8e2441b4 (DEFAULT_TABLE table=test-table [id=4383c5a390d742ccb81184e596b01954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:22:33.628396 12702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55171a9ea959456b8a4b311a8e2441b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:22:33.629045 12832 tablet_service.cc:1467] Processing CreateTablet for tablet 55171a9ea959456b8a4b311a8e2441b4 (DEFAULT_TABLE table=test-table [id=4383c5a390d742ccb81184e596b01954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:22:33.630198 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55171a9ea959456b8a4b311a8e2441b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:22:33.639328 13063 tablet_bootstrap.cc:492] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Bootstrap starting.
I20260619 22:22:33.639427 13061 tablet_bootstrap.cc:492] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Bootstrap starting.
I20260619 22:22:33.642784 13061 tablet_bootstrap.cc:654] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Neither blocks nor log segments found. Creating new log.
I20260619 22:22:33.644533 13061 log.cc:824] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Log is configured to *not* fsync() on all Append() calls
I20260619 22:22:33.647617 13061 tablet_bootstrap.cc:492] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: No bootstrap required, opened a new log
I20260619 22:22:33.647878 13061 ts_tablet_manager.cc:1397] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 22:22:33.651192 13062 tablet_bootstrap.cc:492] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: Bootstrap starting.
I20260619 22:22:33.651927 13063 tablet_bootstrap.cc:654] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Neither blocks nor log segments found. Creating new log.
I20260619 22:22:33.653483 13063 log.cc:824] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Log is configured to *not* fsync() on all Append() calls
I20260619 22:22:33.656147 13062 tablet_bootstrap.cc:654] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: Neither blocks nor log segments found. Creating new log.
I20260619 22:22:33.655921 13061 raft_consensus.cc:348] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.656258 13061 raft_consensus.cc:374] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:22:33.656368 13061 raft_consensus.cc:729] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b62b04306540789c2adceb0d4c2664, State: Initialized, Role: FOLLOWER
I20260619 22:22:33.657087 13061 consensus_queue.cc:260] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.657301 13062 log.cc:824] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: Log is configured to *not* fsync() on all Append() calls
I20260619 22:22:33.658407 13061 ts_tablet_manager.cc:1428] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260619 22:22:33.658906 13042 heartbeater.cc:503] Master 127.1.74.190:36791 was elected leader, sending a full tablet report...
I20260619 22:22:33.659914 13063 tablet_bootstrap.cc:492] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: No bootstrap required, opened a new log
I20260619 22:22:33.660128 13063 ts_tablet_manager.cc:1397] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
Jun 19 22:22:33 dist-test-slave-k84h krb5kdc[12082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907752, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:22:33.665854 13062 tablet_bootstrap.cc:492] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: No bootstrap required, opened a new log
I20260619 22:22:33.666193 13062 ts_tablet_manager.cc:1397] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260619 22:22:33.674993 13062 raft_consensus.cc:348] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.675382 13062 raft_consensus.cc:374] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:22:33.675472 13062 raft_consensus.cc:729] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf05885e75f04ff0b4d5563cc5cd09e7, State: Initialized, Role: FOLLOWER
I20260619 22:22:33.675230 13063 raft_consensus.cc:348] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.675554 13063 raft_consensus.cc:374] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:22:33.675627 13063 raft_consensus.cc:729] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6334e50a9934772b37e38ad303cd784, State: Initialized, Role: FOLLOWER
I20260619 22:22:33.676060 13062 consensus_queue.cc:260] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.676105 13063 consensus_queue.cc:260] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.677332 13062 ts_tablet_manager.cc:1428] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20260619 22:22:33.677781 12771 heartbeater.cc:503] Master 127.1.74.190:36791 was elected leader, sending a full tablet report...
I20260619 22:22:33.678929 13063 ts_tablet_manager.cc:1428] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260619 22:22:33.679647 12908 heartbeater.cc:503] Master 127.1.74.190:36791 was elected leader, sending a full tablet report...
W20260619 22:22:33.720732 12772 tablet.cc:2369] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:22:33.782334 13043 tablet.cc:2369] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:22:33.844769 12909 tablet.cc:2369] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:22:33.859860 13066 raft_consensus.cc:482] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:22:33.860110 13066 raft_consensus.cc:504] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.861531 13066 leader_election.cc:288] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf05885e75f04ff0b4d5563cc5cd09e7 (127.1.74.129:44291), e6334e50a9934772b37e38ad303cd784 (127.1.74.130:40825)
I20260619 22:22:33.867466 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.862016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:44291 (local address 127.1.74.131:54289)
0619 22:22:33.862354 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.862370 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.862467 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.862803 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.862806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.862825 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:22:33.862977 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.862989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.864922 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.864929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.866209 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.866216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.867032 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.867036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.867137 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.867159 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.867246 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.867323 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260619 22:22:33.867466 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.862314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:54289 (local address 127.1.74.129:44291)
0619 22:22:33.862595 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.862599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.862611 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:22:33.862654 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.862656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.862693 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.862792 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:22:33.863976 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.864787 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.866337 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.866920 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.867045 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.867102 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.867273 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.867346 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260619 22:22:33.868232 12710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55171a9ea959456b8a4b311a8e2441b4" candidate_uuid: "66b62b04306540789c2adceb0d4c2664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" is_pre_election: true
I20260619 22:22:33.868589 12710 raft_consensus.cc:2454] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b62b04306540789c2adceb0d4c2664 in term 0.
I20260619 22:22:33.869163 12930 leader_election.cc:302] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b62b04306540789c2adceb0d4c2664, cf05885e75f04ff0b4d5563cc5cd09e7; no voters:
I20260619 22:22:33.869503 13066 raft_consensus.cc:2781] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:22:33.869586 13066 raft_consensus.cc:482] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:22:33.869635 13066 raft_consensus.cc:3037] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:22:33.871165 13066 raft_consensus.cc:504] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.871417 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.863553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:41407 (local address 127.1.74.130:40825)
0619 22:22:33.863676 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.863680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.863690 (+ 10us) server_negotiation.cc:407] Connection header received
0619 22:22:33.863732 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.863735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.863773 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.863858 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:22:33.865227 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.865958 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.869954 (+ 3996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.870592 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.870693 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.870759 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.871194 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.871293 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:22:33.871587 13066 leader_election.cc:288] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [CANDIDATE]: Term 1 election: Requested vote from peers cf05885e75f04ff0b4d5563cc5cd09e7 (127.1.74.129:44291), e6334e50a9934772b37e38ad303cd784 (127.1.74.130:40825)
I20260619 22:22:33.872072 12710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55171a9ea959456b8a4b311a8e2441b4" candidate_uuid: "66b62b04306540789c2adceb0d4c2664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7"
I20260619 22:22:33.872200 12710 raft_consensus.cc:3037] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:22:33.873263 12710 raft_consensus.cc:2454] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b62b04306540789c2adceb0d4c2664 in term 1.
I20260619 22:22:33.873626 12930 leader_election.cc:302] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b62b04306540789c2adceb0d4c2664, cf05885e75f04ff0b4d5563cc5cd09e7; no voters:
I20260619 22:22:33.873754 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.863118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:40825 (local address 127.1.74.131:41407)
0619 22:22:33.863409 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.863424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:22:33.863528 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:22:33.864955 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:22:33.864957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:22:33.864977 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:22:33.865088 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.865093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.866093 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.866096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.869775 (+ 3679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:22:33.869810 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.870900 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.870904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:22:33.871026 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.871058 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 22:22:33.871892 (+ 834us) client_negotiation.cc:240] Negotiation successful
0619 22:22:33.871972 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":3,"thread_start_us":138,"threads_started":1}
I20260619 22:22:33.873818 13066 raft_consensus.cc:2781] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:22:33.874182 13066 raft_consensus.cc:686] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 1 LEADER]: Becoming Leader. State: Replica: 66b62b04306540789c2adceb0d4c2664, State: Running, Role: LEADER
I20260619 22:22:33.874607 13066 consensus_queue.cc:237] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } }
I20260619 22:22:33.874732 12852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55171a9ea959456b8a4b311a8e2441b4" candidate_uuid: "66b62b04306540789c2adceb0d4c2664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6334e50a9934772b37e38ad303cd784" is_pre_election: true
I20260619 22:22:33.875025 12852 raft_consensus.cc:2454] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b62b04306540789c2adceb0d4c2664 in term 0.
I20260619 22:22:33.875388 12844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55171a9ea959456b8a4b311a8e2441b4" candidate_uuid: "66b62b04306540789c2adceb0d4c2664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6334e50a9934772b37e38ad303cd784"
I20260619 22:22:33.875491 12844 raft_consensus.cc:3037] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:22:33.876780 12844 raft_consensus.cc:2454] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b62b04306540789c2adceb0d4c2664 in term 1.
I20260619 22:22:33.878075 12556 catalog_manager.cc:5495] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66b62b04306540789c2adceb0d4c2664 (127.1.74.131). New cstate: current_term: 1 leader_uuid: "66b62b04306540789c2adceb0d4c2664" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b62b04306540789c2adceb0d4c2664" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 } health_report { overall_health: UNKNOWN } } }
I20260619 22:22:33.906220 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.900701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.1.74.131:45731)
0619 22:22:33.901098 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 22:22:33.901103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:22:33.901117 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:22:33.901205 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:22:33.901208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:22:33.901275 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:22:33.901398 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:22:33.901656 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.902165 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.903360 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:22:33.903708 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:22:33.903823 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:22:33.904290 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:22:33.904921 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:22:33.905042 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 22:22:33.905994 (+ 952us) server_negotiation.cc:299] Negotiation successful
0619 22:22:33.906067 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260619 22:22:33.913878 12844 raft_consensus.cc:1261] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 1 FOLLOWER]: Refusing update from remote peer 66b62b04306540789c2adceb0d4c2664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:22:33.914695 13066 consensus_queue.cc:1035] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6334e50a9934772b37e38ad303cd784" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 40825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:22:33.922431 12710 raft_consensus.cc:1261] T 55171a9ea959456b8a4b311a8e2441b4 P cf05885e75f04ff0b4d5563cc5cd09e7 [term 1 FOLLOWER]: Refusing update from remote peer 66b62b04306540789c2adceb0d4c2664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:22:33.923290 13072 consensus_queue.cc:1035] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf05885e75f04ff0b4d5563cc5cd09e7" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:22:33.926080 13075 mvcc.cc:205] Tried to move back new op lower bound from 7298694160022147072 to 7298694159873556480. Current Snapshot: MvccSnapshot[applied={T|T < 7298694160022147072}]
I20260619 22:22:33.934351 13087 mvcc.cc:205] Tried to move back new op lower bound from 7298694160022147072 to 7298694159873556480. Current Snapshot: MvccSnapshot[applied={T|T < 7298694160022147072}]
I20260619 22:22:33.937577 12556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:22:33.937786 12556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:22:33.940629 12556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa7cf44b7ba14ab6926452b1678f433b: Sending DeleteTablet for 3 replicas of tablet 55171a9ea959456b8a4b311a8e2441b4
I20260619 22:22:33.941699 12702 tablet_service.cc:1514] Processing DeleteTablet for tablet 55171a9ea959456b8a4b311a8e2441b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:22:33 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:54934
I20260619 22:22:33.941784 12975 tablet_service.cc:1514] Processing DeleteTablet for tablet 55171a9ea959456b8a4b311a8e2441b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:22:33 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:59688
I20260619 22:22:33.942286 12832 tablet_service.cc:1514] Processing DeleteTablet for tablet 55171a9ea959456b8a4b311a8e2441b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:22:33 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53646
I20260619 22:22:33.943005 13093 tablet_replica.cc:330] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: stopping tablet replica
I20260619 22:22:33.943384 13093 raft_consensus.cc:2229] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:22:33.943537 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 12639
I20260619 22:22:33.942821 13092 tablet_replica.cc:330] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: stopping tablet replica
I20260619 22:22:33.944525 13092 raft_consensus.cc:2229] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:22:33.944962 13093 raft_consensus.cc:2258] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:22:33.945554 13092 raft_consensus.cc:2258] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:22:33.948506 13093 ts_tablet_manager.cc:1905] T 55171a9ea959456b8a4b311a8e2441b4 P e6334e50a9934772b37e38ad303cd784: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:22:33.951402 13092 ts_tablet_manager.cc:1905] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 22:22:33.953751 12543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:44291 (error 108)
I20260619 22:22:33.954208 13092 ts_tablet_manager.cc:1918] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:22:33.954315 13092 log.cc:1196] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/55171a9ea959456b8a4b311a8e2441b4
I20260619 22:22:33.954649 13092 ts_tablet_manager.cc:1939] T 55171a9ea959456b8a4b311a8e2441b4 P 66b62b04306540789c2adceb0d4c2664: Deleting consensus metadata
I20260619 22:22:33.954792 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 12778
I20260619 22:22:33.956894 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.956531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:44291 (local address 127.0.0.1:54944)
0619 22:22:33.956723 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.956788 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:44291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260619 22:22:33.957165 12543 catalog_manager.cc:4584] TS cf05885e75f04ff0b4d5563cc5cd09e7 (127.1.74.129:44291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55171a9ea959456b8a4b311a8e2441b4: Network error: Client connection negotiation failed: client connection to 127.1.74.129:44291: connect: Connection refused (error 111)
I20260619 22:22:33.960295 12545 catalog_manager.cc:4841] TS 66b62b04306540789c2adceb0d4c2664 (127.1.74.131:45731): tablet 55171a9ea959456b8a4b311a8e2441b4 (table test-table [id=4383c5a390d742ccb81184e596b01954]) successfully deleted
I20260619 22:22:33.962504 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 12913
I20260619 22:22:33.962584 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:22:33.962286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:40825 (local address 127.0.0.1:53654)
0619 22:22:33.962436 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 22:22:33.962493 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 22:22:33.968894 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 12526
I20260619 22:22:33.977213 1322 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 22:22:34.313133 1322 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 22:22:34.313292 1322 mini_ranger.cc:66] Stopping Ranger...
I20260619 22:22:34.658937 1322 mini_ranger.cc:68] Stopped Ranger
2026-06-19 22:22:34.659 UTC [12112] LOG: received smart shutdown request
2026-06-19 22:22:34.663 UTC [12112] LOG: background worker "logical replication launcher" (PID 12121) exited with exit code 1
2026-06-19 22:22:34.663 UTC [12116] LOG: shutting down
2026-06-19 22:22:34.848 UTC [12112] LOG: database system is shut down
2026-06-19T22:22:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46951 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:22:35 dist-test-slave-k84h krb5kdc[13103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:22:35 dist-test-slave-k84h krb5kdc[13103](info): set up 2 sockets
Jun 19 22:22:35 dist-test-slave-k84h krb5kdc[13103](info): commencing operation
krb5kdc: starting...
W20260619 22:22:36.027117 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:22:36 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:22:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:22:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/rangeradmin_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/rangeradmin_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/rangerlookup_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/rangerlookup_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
I20260619 22:22:36.214214 1322 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSWejqq/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/postgres -l logfile start
2026-06-19 22:22:38.888 UTC [13133] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 22:22:38.888 UTC [13133] LOG: listening on IPv4 address "127.1.74.148", port 43619
2026-06-19 22:22:38.892 UTC [13133] LOG: listening on Unix socket "/tmp/.s.PGSQL.43619"
2026-06-19 22:22:38.913 UTC [13136] LOG: database system was shut down at 2026-06-19 22:22:37 UTC
2026-06-19 22:22:38.916 UTC [13133] LOG: database system is ready to accept connections
I20260619 22:22:39.867167 1322 mini_postgres.cc:96] Postgres bound to 43619
2026-06-19 22:22:39.873 UTC [13144] FATAL: database "slave" does not exist
127.1.74.148:43619 - accepting connections
I20260619 22:22:39.874540 1322 mini_ranger.cc:161] Starting Ranger...
I20260619 22:22:39.890831 1322 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 22:22:40.437697 1322 mini_ranger.cc:87] Created ranger Postgres database
I20260619 22:22:40.437783 1322 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-admin
2026-06-19 22:22:40,735 [I] DB FLAVOR :POSTGRES
2026-06-19 22:22:40,736 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 22:22:40,736 [I] Checking connection..
2026-06-19 22:22:40,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 22:22:40,995 [I] Checking connection passed.
2026-06-19 22:22:40,995 [I] --------- Verifying version history table ---------
2026-06-19 22:22:40,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 22:22:41,265 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 22:22:41,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 22:22:41,523 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 22:22:41,523 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 22:22:41,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 22:22:41.778 UTC [13208] WARNING: there is no transaction in progress
2026-06-19 22:22:41,785 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 22:22:41,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 22:22:42,040 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 22:22:42,041 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 22:22:42,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 22:22:42,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 22:22:42,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-k84h.c.gcp-upstream.internal','N') ;"
2026-06-19 22:22:42,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 22:22:43,058 [I] Table x_portal_user does not exist in database ranger
2026-06-19 22:22:43,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 22:22:43,311 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 22:22:43,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 22:22:43,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 22:22:43,818 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 22:22:43,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 22:22:44.885 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:44.904 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:44.924 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:44.943 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:44.963 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:44.986 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.055 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.064 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.081 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.095 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.111 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.129 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.141 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.149 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.157 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45.165 UTC [13316] WARNING: there is no transaction in progress
2026-06-19 22:22:45,852 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 22:22:45,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 22:22:46,109 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 22:22:46,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 22:22:46,373 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 22:22:46,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 22:22:46,655 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 22:22:46,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 22:22:46,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 22:22:47,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-k84h.c.gcp-upstream.internal';"
2026-06-19 22:22:47,435 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 22:22:47,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 22:22:47,689 [I] DB_PATCHES have already been applied
I20260619 22:22:47.696502 1322 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSWejqq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 22:22:47.696638 1322 mini_ranger.cc:193] Using host: 127.1.74.148
I20260619 22:22:47.701361 1322 mini_ranger.cc:227] Ranger admin URL: http://127.1.74.148:34177
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 10:22:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 22:22:48.852506 1322 mini_ranger.cc:160] Time spent starting Ranger: real 8.978s user 0.000s sys 0.012s
Jun 19, 2026 10:23:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 10:23:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Jun 19, 2026 10:23:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 10:23:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 10:23:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 22:23:16.887517 1322 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.74.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.74.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/rangerkms_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.74.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/rangerkms_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.74.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.74.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.74.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/HTTP_127.1.74.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 22:23:16 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907796, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 22:23:16.979576 1322 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 22:23:16.991832 1322 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 22:23:17.963617 1322 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 22:23:17.963711 1322 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms
I20260619 22:23:17.963734 1322 mini_ranger_kms.cc:227] Using postgres at 127.1.74.148:43619
2026-06-19 22:23:18,134 [I] DB FLAVOR :POSTGRES
2026-06-19 22:23:18,134 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 22:23:18,134 [I] Checking connection
2026-06-19 22:23:18,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 22:23:18,391 [I] connection success
2026-06-19 22:23:18,391 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 22:23:18,392 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 22:23:18,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 22:23:18,637 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 22:23:18,638 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 22:23:18,638 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 22:23:18,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.74.148:43619/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 22:23:18,929 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 22:23:19.428121 1322 mini_ranger_kms.cc:310] Created kms service
I20260619 22:23:19.603200 1322 mini_ranger_kms.cc:326] Created kudu user
I20260619 22:23:19.651729 1322 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 22:23:19 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907796, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:20.071400 1322 mini_ranger_kms.cc:384] Added ranger policy
I20260619 22:23:20.071565 1322 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSWejqq/build/debug/bin/postgresql.jar:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSWejqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 22:23:20.071601 1322 mini_ranger_kms.cc:240] Using host: 127.1.74.148
I20260619 22:23:20.075806 1322 mini_ranger_kms.cc:276] Ranger KMS PID: 13497
I20260619 22:23:20.075928 1322 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.74.148:56957
Jun 19, 2026 10:23:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:23:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:23:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 10:23:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 10:23:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 10:23:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 22:23:22 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781907802, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.74.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:22.341461 1322 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.362s user 0.011s sys 0.144s
I20260619 22:23:22.341568 1322 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 22:23:22.341653 1322 mini_ranger_kms.cc:401] 127.1.74.148:56957/kms/v1/keys
Jun 19 22:23:22 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907802, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.74.148@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
Jun 19 22:23:23 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907802, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.74.148@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
Jun 19 22:23:25 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:23:25.412461 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:43275
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:43275
--ranger_config_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:25.527355 13547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:25.527645 13547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:25.527693 13547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:25.531837 13547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:25.531925 13547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:25.531946 13547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:25.531963 13547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:25.531978 13547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:25.537832 13547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.74.148:56957/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:43275
--ranger_config_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:43275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:25.538915 13547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:25.539920 13547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:25.549345 13553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:25.549500 13555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:25.549358 13552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:25.549320 13547 server_base.cc:1034] running on GCE node
I20260619 22:23:25.550264 13547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:25.551535 13547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:25.552747 13547 hybrid_clock.cc:648] HybridClock initialized: now 1781907805552724 us; error 47 us; skew 500 ppm
Jun 19 22:23:25 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907805, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:25.555728 13547 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:25.556706 13547 webserver.cc:460] Webserver started at http://127.1.74.190:38879/ using document root <none> and password file <none>
I20260619 22:23:25.557286 13547 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:25.557381 13547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:25.557613 13547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:23:25 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907805, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:25.608753 13547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "23ce497ff98a4c38861a0711dfdbbeba"
format_stamp: "Formatted at 2026-06-19 22:23:25 on dist-test-slave-k84h"
server_key: "38697bb1ec1e2ee17cd92085a4b76203"
server_key_iv: "3335bb931f5f767c41a99194503e8226"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:25.609517 13547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "23ce497ff98a4c38861a0711dfdbbeba"
format_stamp: "Formatted at 2026-06-19 22:23:25 on dist-test-slave-k84h"
server_key: "38697bb1ec1e2ee17cd92085a4b76203"
server_key_iv: "3335bb931f5f767c41a99194503e8226"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:25.614487 13547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 22:23:25.629930 13562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:25.631518 13547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 22:23:25.631686 13547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "23ce497ff98a4c38861a0711dfdbbeba"
format_stamp: "Formatted at 2026-06-19 22:23:25 on dist-test-slave-k84h"
server_key: "38697bb1ec1e2ee17cd92085a4b76203"
server_key_iv: "3335bb931f5f767c41a99194503e8226"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:25.631855 13547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:25.654263 13547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:25.657466 13547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:25.657709 13547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:25.665343 13547 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:43275
I20260619 22:23:25.665350 13614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:43275 every 8 connection(s)
I20260619 22:23:25.666347 13547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:25.669427 13615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:25.671018 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 13547
I20260619 22:23:25.671156 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:23:25.671453 1322 external_mini_cluster.cc:1428] Setting key 1243519bc63404cb56f30aaf8e9d4829
I20260619 22:23:25.676204 13615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Bootstrap starting.
I20260619 22:23:25.678519 13615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:25.679554 13615 log.cc:824] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:23:25 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:25.681843 13615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: No bootstrap required, opened a new log
I20260619 22:23:25.685269 13615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:25.685547 13615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:25.685626 13615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23ce497ff98a4c38861a0711dfdbbeba, State: Initialized, Role: FOLLOWER
I20260619 22:23:25.686158 13615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:25.686363 13615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:25.686457 13615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:25.686555 13615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:25.687197 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:25.672897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.1.74.190:43275)
0619 22:23:25.673441 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:25.673458 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:25.673518 (+ 60us) server_negotiation.cc:407] Connection header received
0619 22:23:25.674550 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:25.674569 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:25.675007 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:25.675351 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:25.676203 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:25.677245 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:25.678119 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:25.678606 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:25.678795 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:25.681713 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:25.681719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:25.681733 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:25.681759 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:25.684077 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:25.684590 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:25.684596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:25.684601 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:25.684711 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:25.685015 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:25.685019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:25.685021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:25.685450 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:25.685687 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:25.685959 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 22:23:25.686254 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":146,"threads_started":1}
I20260619 22:23:25.687767 13615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:25.688117 13615 leader_election.cc:302] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23ce497ff98a4c38861a0711dfdbbeba; no voters:
I20260619 22:23:25.688508 13615 leader_election.cc:288] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:23:25.688591 13620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:25.688795 13620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 LEADER]: Becoming Leader. State: Replica: 23ce497ff98a4c38861a0711dfdbbeba, State: Running, Role: LEADER
I20260619 22:23:25.689139 13620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:25.689836 13615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:25.690418 13620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ce497ff98a4c38861a0711dfdbbeba. Latest consensus state: current_term: 1 leader_uuid: "23ce497ff98a4c38861a0711dfdbbeba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } } }
I20260619 22:23:25.690536 13620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:25.691607 13621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23ce497ff98a4c38861a0711dfdbbeba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } } }
I20260619 22:23:25.691716 13621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:25.692209 13615 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 22:23:25.693346 13626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:25.698446 13626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:25.704974 13626 catalog_manager.cc:1347] Generated new cluster ID: 99775eb932fb4f7f88112032f7ae3c05
I20260619 22:23:25.705161 13626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:25.726749 13626 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:23:25.728194 13626 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:25.749019 13626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Generated new TSK 0
I20260619 22:23:25.749923 13626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:23:26.422086 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:43275
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:26.580325 13660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:26.580787 13660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:26.580885 13660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:26.588121 13660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:26.588227 13660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:26.588326 13660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:23:26.597517 13660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.74.148:56957/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:26.599203 13660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:26.600667 13660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:26.613852 13668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:26.613835 13667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:26.613885 13670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:26.615455 13660 server_base.cc:1034] running on GCE node
I20260619 22:23:26.616024 13660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:26.616961 13660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:26.618160 13660 hybrid_clock.cc:648] HybridClock initialized: now 1781907806618126 us; error 58 us; skew 500 ppm
Jun 19 22:23:26 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:26.622601 13660 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:23:26.624154 13660 webserver.cc:460] Webserver started at http://127.1.74.129:39341/ using document root <none> and password file <none>
I20260619 22:23:26.625079 13660 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:26.625265 13660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:26.625564 13660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:23:26 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:26.659137 13660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "062e67a6109144048f7c7076071ee7ad"
format_stamp: "Formatted at 2026-06-19 22:23:26 on dist-test-slave-k84h"
server_key: "90b647ed6068d4d2faa5d98ec5467945"
server_key_iv: "9678e90afeadeb825f4a138a7df5295d"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:26.659938 13660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "062e67a6109144048f7c7076071ee7ad"
format_stamp: "Formatted at 2026-06-19 22:23:26 on dist-test-slave-k84h"
server_key: "90b647ed6068d4d2faa5d98ec5467945"
server_key_iv: "9678e90afeadeb825f4a138a7df5295d"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:26.665155 13660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 22:23:26.694047 13679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:26.695843 13660 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260619 22:23:26.696007 13660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "062e67a6109144048f7c7076071ee7ad"
format_stamp: "Formatted at 2026-06-19 22:23:26 on dist-test-slave-k84h"
server_key: "90b647ed6068d4d2faa5d98ec5467945"
server_key_iv: "9678e90afeadeb825f4a138a7df5295d"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:26.696170 13660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:26.723627 13660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:26.727020 13660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:26.727224 13660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:26.727877 13660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:26.728806 13660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:26.728884 13660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:26.728952 13660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:26.728981 13660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:26.740656 13660 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:38807
I20260619 22:23:26.740680 13792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:38807 every 8 connection(s)
I20260619 22:23:26.741631 13660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
Jun 19 22:23:26 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:26.747474 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 13660
I20260619 22:23:26.747618 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:23:26.747938 1322 external_mini_cluster.cc:1428] Setting key ba9c6dc74a42fef8d08ff3a4ef6c536f
Jun 19 22:23:26 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:26.761127 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:26.745415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43275 (local address 127.1.74.129:34783)
0619 22:23:26.745898 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:26.745951 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:26.747050 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:26.747802 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:26.747812 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:26.748158 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:26.748901 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:26.748922 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:26.750123 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:26.750126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:26.751622 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:26.751633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:26.752418 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:26.752422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:26.752625 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:26.753356 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:26.753378 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:26.755213 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:26.757624 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:26.757633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:26.757654 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:26.758035 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:26.758399 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:26.758402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:26.758405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:26.758599 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:26.759139 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:26.759146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:26.759493 (+ 347us) client_negotiation.cc:769] Sending connection context
0619 22:23:26.759695 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 22:23:26.760049 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":130,"threads_started":1}
I20260619 22:23:26.761932 13795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:26.744878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:34783 (local address 127.1.74.190:43275)
0619 22:23:26.745168 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:26.745177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:26.746055 (+ 878us) server_negotiation.cc:407] Connection header received
0619 22:23:26.747321 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:26.747325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:26.747393 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:26.747493 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:26.749078 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:26.749966 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:26.751768 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:26.752099 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:26.752217 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:26.755446 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:26.755450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:26.755453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:26.755482 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:26.757474 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:26.758164 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:26.758168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:26.758170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:26.758285 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:26.758717 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:26.758721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:26.758723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:26.758989 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:26.759083 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:26.761593 (+ 2510us) server_negotiation.cc:299] Negotiation successful
0619 22:23:26.761742 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260619 22:23:26.763696 13793 heartbeater.cc:347] Connected to a master server at 127.1.74.190:43275
I20260619 22:23:26.763970 13793 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:26.764516 13793 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:26.766206 13579 ts_manager.cc:194] Registered new tserver with Master: 062e67a6109144048f7c7076071ee7ad (127.1.74.129:38807)
I20260619 22:23:26.767827 13579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:34783
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:23:26.798931 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:43275
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:26.948369 13799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:26.948719 13799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:26.948812 13799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:26.953397 13799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:26.953516 13799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:26.953619 13799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:23:26.961349 13799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.74.148:56957/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:26.962813 13799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:26.964213 13799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:26.976847 13805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:26.976840 13806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:26.983843 13808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:26.984488 13799 server_base.cc:1034] running on GCE node
I20260619 22:23:26.985001 13799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:26.985754 13799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:26.986924 13799 hybrid_clock.cc:648] HybridClock initialized: now 1781907806986903 us; error 42 us; skew 500 ppm
Jun 19 22:23:26 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:26.990082 13799 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:23:26.991309 13799 webserver.cc:460] Webserver started at http://127.1.74.130:35891/ using document root <none> and password file <none>
I20260619 22:23:26.992097 13799 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:26.992189 13799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:26.992439 13799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:23:26 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:27.007990 13799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "b8920704273948059c5f476668da14e6"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "345045ac71f9e96528df27d3c18ee1d0"
server_key_iv: "cb572f41c6a232dab7bacb1a63f05aac"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.008886 13799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "b8920704273948059c5f476668da14e6"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "345045ac71f9e96528df27d3c18ee1d0"
server_key_iv: "cb572f41c6a232dab7bacb1a63f05aac"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.028694 13799 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.004s sys 0.000s
I20260619 22:23:27.055408 13815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.058141 13799 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.000s
I20260619 22:23:27.058341 13799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "b8920704273948059c5f476668da14e6"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "345045ac71f9e96528df27d3c18ee1d0"
server_key_iv: "cb572f41c6a232dab7bacb1a63f05aac"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.058512 13799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:27.075699 13799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:27.079578 13799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:27.079795 13799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:27.080528 13799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:27.081552 13799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:27.081617 13799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.081683 13799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:27.081724 13799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.094400 13799 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:42557
I20260619 22:23:27.098271 13928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:42557 every 8 connection(s)
I20260619 22:23:27.099416 13799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:23:27.108392 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 13799
I20260619 22:23:27.108531 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:23:27.108860 1322 external_mini_cluster.cc:1428] Setting key 1e7a6f865bd3c34f02f50df9eba4cbfa
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:27.122061 13795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:27.102216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:34721 (local address 127.1.74.190:43275)
0619 22:23:27.103033 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:27.103040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:27.104222 (+ 1182us) server_negotiation.cc:407] Connection header received
0619 22:23:27.105652 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:27.105657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:27.105740 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:27.105803 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:27.107761 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.108581 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.110362 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.110749 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.110864 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:27.117371 (+ 6507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:27.117377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:27.117380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:27.117413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:27.119294 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:27.120042 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:27.120047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:27.120048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:27.120147 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:27.120606 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:27.120610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:27.120612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:27.120870 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:27.120960 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:27.121726 (+ 766us) server_negotiation.cc:299] Negotiation successful
0619 22:23:27.121881 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695}
I20260619 22:23:27.124349 13931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:27.102728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43275 (local address 127.1.74.130:34721)
0619 22:23:27.104036 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:27.104100 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:27.105361 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:27.106285 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:27.106296 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:27.106722 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:27.107538 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:27.107561 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.108778 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.108781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:27.110208 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:27.110221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.114020 (+ 3799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.114025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:27.114287 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:27.115110 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:27.115134 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:27.117088 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:27.119452 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:27.119459 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:27.119480 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:27.119900 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:27.120270 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:27.120273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:27.120275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:27.120482 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:27.121034 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:27.121041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:27.121399 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 22:23:27.122787 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0619 22:23:27.123195 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"thread_start_us":149,"threads_started":1}
I20260619 22:23:27.126103 13929 heartbeater.cc:347] Connected to a master server at 127.1.74.190:43275
I20260619 22:23:27.126353 13929 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:27.127063 13929 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:27.128590 13579 ts_manager.cc:194] Registered new tserver with Master: b8920704273948059c5f476668da14e6 (127.1.74.130:42557)
I20260619 22:23:27.129436 13579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:34721
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:23:27.157873 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:43275
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
W20260619 22:23:27.352182 13934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:27.352579 13934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:27.352655 13934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:27.359663 13934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:27.359768 13934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:27.359858 13934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:23:27.368003 13934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.74.148:56957/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:27.369408 13934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:27.370730 13934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:27.384924 13940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:27.385385 13934 server_base.cc:1034] running on GCE node
W20260619 22:23:27.385690 13939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:27.389508 13942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:27.390187 13934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:27.391072 13934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:27.394352 13934 hybrid_clock.cc:648] HybridClock initialized: now 1781907807394271 us; error 88 us; skew 500 ppm
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:27.398206 13934 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:23:27.399468 13934 webserver.cc:460] Webserver started at http://127.1.74.131:36635/ using document root <none> and password file <none>
I20260619 22:23:27.400252 13934 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:27.400395 13934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:27.400671 13934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:27.420836 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "d6b26bfdbc65401989cebf3455dc2a05"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "74323855a400f294be2df4f98def37d0"
server_key_iv: "5334ba9c8d084e3f26bfe046b0ee500e"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.421789 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "d6b26bfdbc65401989cebf3455dc2a05"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "74323855a400f294be2df4f98def37d0"
server_key_iv: "5334ba9c8d084e3f26bfe046b0ee500e"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.426993 13934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 22:23:27.457325 13949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.459170 13934 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260619 22:23:27.459328 13934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "d6b26bfdbc65401989cebf3455dc2a05"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "74323855a400f294be2df4f98def37d0"
server_key_iv: "5334ba9c8d084e3f26bfe046b0ee500e"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.459486 13934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:27.479434 13934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:27.483287 13934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:27.483496 13934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:27.484239 13934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:27.485356 13934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:27.485428 13934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.485495 13934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:27.485534 13934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.505081 13934 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:37789
I20260619 22:23:27.506341 13934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:23:27.507740 14062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:37789 every 8 connection(s)
I20260619 22:23:27.508806 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 13934
I20260619 22:23:27.508936 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:23:27.509238 1322 external_mini_cluster.cc:1428] Setting key 5e18127f8e2ad8be9407ded3a7c51dfa
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:27.530261 13795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:27.512024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:36487 (local address 127.1.74.190:43275)
0619 22:23:27.512283 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:27.512288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:27.514161 (+ 1873us) server_negotiation.cc:407] Connection header received
0619 22:23:27.515589 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:27.515592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:27.515650 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:27.515711 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:27.517497 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.518395 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.520167 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.520535 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.520628 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:27.524006 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:27.524011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:27.524014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:27.524047 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:27.527507 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:27.528254 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:27.528258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:27.528260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:27.528357 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:27.528825 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:27.528829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:27.528831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:27.529071 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:27.529155 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:27.529928 (+ 773us) server_negotiation.cc:299] Negotiation successful
0619 22:23:27.530073 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260619 22:23:27.532063 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:27.510416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43275 (local address 127.1.74.131:36487)
0619 22:23:27.513994 (+ 3578us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:27.514062 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:27.515253 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:27.516160 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:27.516172 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:27.516587 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:27.517303 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:27.517326 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.518568 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.518571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:27.520012 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:27.520025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:27.520864 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:27.520869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:27.521085 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:27.521868 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:27.521891 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:27.523768 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:27.527671 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:27.527678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:27.527699 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:27.528109 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:27.528480 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:27.528483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:27.528485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:27.528703 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:27.529241 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:27.529248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:27.529620 (+ 372us) client_negotiation.cc:769] Sending connection context
0619 22:23:27.530522 (+ 902us) client_negotiation.cc:240] Negotiation successful
0619 22:23:27.530933 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2216,"thread_start_us":180,"threads_started":1}
I20260619 22:23:27.533411 14063 heartbeater.cc:347] Connected to a master server at 127.1.74.190:43275
I20260619 22:23:27.533667 14063 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:27.534308 14063 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:27.535547 13579 ts_manager.cc:194] Registered new tserver with Master: d6b26bfdbc65401989cebf3455dc2a05 (127.1.74.131:37789)
I20260619 22:23:27.536408 13579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:36487
I20260619 22:23:27.539633 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:23:27.540482 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 13660
I20260619 22:23:27.550115 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 13799
I20260619 22:23:27.556456 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 13934
I20260619 22:23:27.563696 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 13547
I20260619 22:23:27.572046 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:43275
--webserver_interface=127.1.74.190
--webserver_port=38879
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:43275
--ranger_config_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:27.717530 14066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:27.717833 14066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:27.717876 14066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:27.722225 14066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:27.722308 14066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:27.722328 14066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:27.722344 14066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:27.722360 14066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:27.728158 14066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:43275
--ranger_config_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:43275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:27.729370 14066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:27.730444 14066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:27.739204 14071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:27.739190 14072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:27.739190 14074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:27.739274 14066 server_base.cc:1034] running on GCE node
I20260619 22:23:27.739934 14066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:27.741029 14066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:27.742194 14066 hybrid_clock.cc:648] HybridClock initialized: now 1781907807742166 us; error 46 us; skew 500 ppm
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:27.745162 14066 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:27.746074 14066 webserver.cc:460] Webserver started at http://127.1.74.190:38879/ using document root <none> and password file <none>
I20260619 22:23:27.746634 14066 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:27.746701 14066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 22:23:27 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:27.827224 14066 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:23:27.829921 14082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:27.831351 14066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:27.831506 14066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "23ce497ff98a4c38861a0711dfdbbeba"
format_stamp: "Formatted at 2026-06-19 22:23:25 on dist-test-slave-k84h"
server_key: "38697bb1ec1e2ee17cd92085a4b76203"
server_key_iv: "3335bb931f5f767c41a99194503e8226"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:27.832058 14066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:27.853253 14066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:27.856353 14066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:27.856586 14066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:27.864593 14066 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:43275
I20260619 22:23:27.864619 14134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:43275 every 8 connection(s)
I20260619 22:23:27.865717 14066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:27.870258 14135 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:23:27.871479 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14066
I20260619 22:23:27.871938 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:38807
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:43275
--webserver_port=39341
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:23:27.872370 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Bootstrap starting.
I20260619 22:23:27.889482 14135 log.cc:824] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:27.896565 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:23:27.897020 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Bootstrap complete.
I20260619 22:23:27.901257 14135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:27.901654 14135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23ce497ff98a4c38861a0711dfdbbeba, State: Initialized, Role: FOLLOWER
I20260619 22:23:27.902197 14135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:27.902314 14135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:27.902398 14135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:27.902545 14135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:23:27.904556 14135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:27.904893 14135 leader_election.cc:302] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23ce497ff98a4c38861a0711dfdbbeba; no voters:
I20260619 22:23:27.905229 14135 leader_election.cc:288] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 22:23:27.905316 14139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 2 FOLLOWER]: Leader election won for term 2
I20260619 22:23:27.905515 14139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [term 2 LEADER]: Becoming Leader. State: Replica: 23ce497ff98a4c38861a0711dfdbbeba, State: Running, Role: LEADER
I20260619 22:23:27.905818 14139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } }
I20260619 22:23:27.906507 14135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:27.907433 14141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ce497ff98a4c38861a0711dfdbbeba. Latest consensus state: current_term: 2 leader_uuid: "23ce497ff98a4c38861a0711dfdbbeba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } } }
I20260619 22:23:27.907562 14141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:27.907486 14140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "23ce497ff98a4c38861a0711dfdbbeba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ce497ff98a4c38861a0711dfdbbeba" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43275 } } }
I20260619 22:23:27.907611 14140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:27.908988 14135 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 22:23:27.909389 14148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:27.915885 14148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:27.916723 14148 catalog_manager.cc:1259] Loaded cluster ID: 99775eb932fb4f7f88112032f7ae3c05
I20260619 22:23:27.916790 14148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:27.919435 14148 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:27.920375 14148 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Loaded TSK: 0
I20260619 22:23:27.921741 14148 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 22:23:27.998973 14137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:27.999269 14137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:27.999344 14137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:28.004173 14137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:28.004252 14137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:28.004344 14137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:23:28.010586 14137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:38807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=39341
--tserver_master_addrs=127.1.74.190:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:28.011895 14137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:28.012984 14137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:28.025533 14172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:28.025547 14173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:28.025902 14175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:28.026153 14137 server_base.cc:1034] running on GCE node
I20260619 22:23:28.026660 14137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:28.027576 14137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:28.028800 14137 hybrid_clock.cc:648] HybridClock initialized: now 1781907808028779 us; error 36 us; skew 500 ppm
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:28.032655 14137 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:23:28.034014 14137 webserver.cc:460] Webserver started at http://127.1.74.129:39341/ using document root <none> and password file <none>
I20260619 22:23:28.034794 14137 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:28.034945 14137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:28.069548 14137 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.003s sys 0.000s
I20260619 22:23:28.074623 14183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.080049 14137 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260619 22:23:28.080312 14137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "062e67a6109144048f7c7076071ee7ad"
format_stamp: "Formatted at 2026-06-19 22:23:26 on dist-test-slave-k84h"
server_key: "90b647ed6068d4d2faa5d98ec5467945"
server_key_iv: "9678e90afeadeb825f4a138a7df5295d"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:28.081141 14137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:28.124075 14137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:28.127851 14137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:28.128055 14137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:28.128732 14137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:28.129685 14137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:28.129745 14137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.129799 14137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:28.129822 14137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.141527 14137 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:38807
I20260619 22:23:28.141713 14296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:38807 every 8 connection(s)
I20260619 22:23:28.142930 14137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:23:28.152758 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14137
I20260619 22:23:28.153319 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:42557
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:43275
--webserver_port=35891
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:28.166875 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.148683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43275 (local address 127.1.74.129:47819)
0619 22:23:28.149233 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:28.149286 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:28.150221 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:28.151578 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:28.151588 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:28.151963 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:28.152764 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:28.152786 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.154765 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.154769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:28.156197 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:28.156211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.157557 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.157561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:28.157781 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.158513 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:28.158534 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:28.160167 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:28.163009 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:28.163016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:28.163044 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:28.163403 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:28.163852 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:28.163855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:28.163857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:28.164046 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:28.164915 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:28.164922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:28.165221 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 22:23:28.165429 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 22:23:28.165790 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":122,"threads_started":1}
I20260619 22:23:28.168680 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.148542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:47819 (local address 127.1.74.190:43275)
0619 22:23:28.149754 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:28.149776 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:28.149823 (+ 47us) server_negotiation.cc:407] Connection header received
0619 22:23:28.150659 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:28.150669 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:28.151037 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:28.151414 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:28.152994 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.154577 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.156354 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.157125 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.157317 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.160423 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:28.160430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:28.160451 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:28.160487 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:28.162801 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.163553 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.163560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.163565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.163720 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.164164 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.164167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.164169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.164710 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:28.164856 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:28.167449 (+ 2593us) server_negotiation.cc:299] Negotiation successful
0619 22:23:28.167771 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":136,"threads_started":1}
I20260619 22:23:28.172492 14297 heartbeater.cc:347] Connected to a master server at 127.1.74.190:43275
I20260619 22:23:28.172768 14297 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.173388 14297 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
W20260619 22:23:28.175884 14297 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:43275 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 22:23:28.176007 14297 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.176195 14297 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:28.177201 14297 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.177384 14297 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
W20260619 22:23:28.178231 14297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:23:28.319051 14301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:28.319341 14301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:28.319388 14301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:28.325117 14301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:28.325227 14301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:28.325320 14301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:23:28.331352 14301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:42557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=35891
--tserver_master_addrs=127.1.74.190:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:28.332619 14301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:28.333652 14301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:28.343813 14307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:28.344646 14309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:28.344626 14306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:28.345777 14301 server_base.cc:1034] running on GCE node
I20260619 22:23:28.346273 14301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:28.347133 14301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:28.348330 14301 hybrid_clock.cc:648] HybridClock initialized: now 1781907808348298 us; error 51 us; skew 500 ppm
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:28.351715 14301 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:23:28.352847 14301 webserver.cc:460] Webserver started at http://127.1.74.130:35891/ using document root <none> and password file <none>
I20260619 22:23:28.353540 14301 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:28.353637 14301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:28.384508 14301 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260619 22:23:28.387823 14317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.389402 14301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:28.389557 14301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "b8920704273948059c5f476668da14e6"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "345045ac71f9e96528df27d3c18ee1d0"
server_key_iv: "cb572f41c6a232dab7bacb1a63f05aac"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:28.390328 14301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:28.435273 14301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:28.438657 14301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:28.438944 14301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:28.439625 14301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:28.440610 14301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:28.440694 14301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.440814 14301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:28.440874 14301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.454932 14301 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:42557
I20260619 22:23:28.455104 14431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:42557 every 8 connection(s)
I20260619 22:23:28.456363 14301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:23:28.463650 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14301
I20260619 22:23:28.464190 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.74.148:56957/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:37789
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:43275
--webserver_port=36635
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:28.472697 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.458591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:40939 (local address 127.1.74.190:43275)
0619 22:23:28.458780 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:28.458785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:28.459691 (+ 906us) server_negotiation.cc:407] Connection header received
0619 22:23:28.460658 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:28.460661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:28.460736 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:28.460856 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:28.461992 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.462835 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.464093 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.464394 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.464512 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.467688 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:28.467693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:28.467696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:28.467740 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:28.469650 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.470408 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.470414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.470417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.470515 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.471042 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.471048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.471051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.471329 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:28.471413 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:28.472263 (+ 850us) server_negotiation.cc:299] Negotiation successful
0619 22:23:28.472438 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:23:28.474936 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.459037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43275 (local address 127.1.74.130:40939)
0619 22:23:28.459513 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:28.459565 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:28.460432 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:28.461018 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:28.461025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:28.461300 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:28.461817 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:28.461832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.462985 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.462988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:28.463968 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:28.463977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.464697 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.464701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:28.464864 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.465516 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:28.465534 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:28.467464 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:28.469799 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:28.469809 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:28.469832 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:28.470267 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:28.470651 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:28.470656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:28.470661 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:28.470912 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:28.471506 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:28.471517 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:28.471926 (+ 409us) client_negotiation.cc:769] Sending connection context
0619 22:23:28.473185 (+ 1259us) client_negotiation.cc:240] Negotiation successful
0619 22:23:28.473611 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":137,"threads_started":1}
I20260619 22:23:28.476665 14432 heartbeater.cc:347] Connected to a master server at 127.1.74.190:43275
I20260619 22:23:28.476938 14432 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.477468 14432 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
W20260619 22:23:28.479139 14432 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:43275 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 22:23:28.479261 14432 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.479434 14432 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:28.480374 14432 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.480567 14432 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
W20260619 22:23:28.481395 14432 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:23:28.608534 14435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:28.608958 14435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:28.609050 14435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:28.616019 14435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:28.616125 14435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:28.616240 14435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:23:28.625145 14435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.74.148:56957/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:37789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=36635
--tserver_master_addrs=127.1.74.190:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:28.626546 14435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:28.628055 14435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:28.640370 14441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:28.640661 14440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:28.640394 14443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:28.641012 14435 server_base.cc:1034] running on GCE node
I20260619 22:23:28.641578 14435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:28.642526 14435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:28.643733 14435 hybrid_clock.cc:648] HybridClock initialized: now 1781907808643702 us; error 47 us; skew 500 ppm
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:28.647603 14435 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:23:28.648875 14435 webserver.cc:460] Webserver started at http://127.1.74.131:36635/ using document root <none> and password file <none>
I20260619 22:23:28.649607 14435 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:28.649708 14435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:28.670832 14435 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:23:28.674171 14451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.675930 14435 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 22:23:28.676065 14435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "d6b26bfdbc65401989cebf3455dc2a05"
format_stamp: "Formatted at 2026-06-19 22:23:27 on dist-test-slave-k84h"
server_key: "74323855a400f294be2df4f98def37d0"
server_key_iv: "5334ba9c8d084e3f26bfe046b0ee500e"
server_key_version: "kuduclusterkey@0"
I20260619 22:23:28.676815 14435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:28.726603 14435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:28.730602 14435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:28.730829 14435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:28.731627 14435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:28.732694 14435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:28.732762 14435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.732824 14435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:28.732858 14435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:28.744526 14435 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:37789
I20260619 22:23:28.744545 14571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:37789 every 8 connection(s)
I20260619 22:23:28.745927 14435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:23:28.754029 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14435
Jun 19 22:23:28 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907808, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:28.760862 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.748107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:39647 (local address 127.1.74.190:43275)
0619 22:23:28.748266 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:28.748272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:28.749293 (+ 1021us) server_negotiation.cc:407] Connection header received
0619 22:23:28.750228 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:28.750232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:28.750294 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:28.750410 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:28.751610 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.752148 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.753296 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.753524 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.753637 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.756592 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:28.756602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:28.756604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:28.756627 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:28.758484 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.759067 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.759071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.759073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.759177 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.759604 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.759607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.759609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.759833 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:28.759928 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:28.760616 (+ 688us) server_negotiation.cc:299] Negotiation successful
0619 22:23:28.760741 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:23:28.761915 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.748629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43275 (local address 127.1.74.131:39647)
0619 22:23:28.749136 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:28.749178 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:28.750013 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:28.750608 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:28.750624 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:28.750962 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:28.751453 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:28.751466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.752273 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.752278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:28.753160 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:28.753169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.753699 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.753703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:28.753864 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.754504 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:28.754534 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:28.756371 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:28.758605 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:28.758611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:28.758632 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:28.758968 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:28.759277 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:28.759282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:28.759284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:28.759478 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:28.759983 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:28.759990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:28.760307 (+ 317us) client_negotiation.cc:769] Sending connection context
0619 22:23:28.760515 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 22:23:28.760860 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":135,"threads_started":1}
I20260619 22:23:28.763175 14572 heartbeater.cc:347] Connected to a master server at 127.1.74.190:43275
I20260619 22:23:28.763430 14572 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:28.763998 14572 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:28.765447 14099 ts_manager.cc:194] Registered new tserver with Master: d6b26bfdbc65401989cebf3455dc2a05 (127.1.74.131:37789)
I20260619 22:23:28.766314 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:28.754769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.1.74.190:43275)
0619 22:23:28.755182 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:28.755192 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:28.755295 (+ 103us) server_negotiation.cc:407] Connection header received
0619 22:23:28.755467 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:28.755471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:28.755522 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:28.755619 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:28.756504 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.757300 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.758581 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:28.758981 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:28.759098 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:28.762470 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:28.762474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:28.762476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:28.762503 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:28.764462 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.764975 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.764999 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.765001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.765094 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:28.765417 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:28.765421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:28.765423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:28.765634 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:28.765815 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:28.766016 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 22:23:28.766144 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"spinlock_wait_cycles":27520,"thread_start_us":154,"threads_started":1}
I20260619 22:23:28.767040 14099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:39647
Jun 19 22:23:29 dist-test-slave-k84h krb5kdc[13103](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781907809, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:29.178506 14297 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:29.178931 14297 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:29.179947 14099 ts_manager.cc:194] Registered new tserver with Master: 062e67a6109144048f7c7076071ee7ad (127.1.74.129:38807)
I20260619 22:23:29.180797 14099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:47819
I20260619 22:23:29.481626 14432 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:29.481932 14432 heartbeater.cc:511] Master 127.1.74.190:43275 requested a full tablet report, sending...
I20260619 22:23:29.482605 14099 ts_manager.cc:194] Registered new tserver with Master: b8920704273948059c5f476668da14e6 (127.1.74.130:42557)
I20260619 22:23:29.483404 14099 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:40939
Jun 19 22:23:29 dist-test-slave-k84h krb5kdc[13103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907809, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for HTTP/127.1.74.148@KRBTEST.COM
I20260619 22:23:29.494906 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:23:29.507499 14586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.496615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.1.74.190:43275)
0619 22:23:29.496919 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.496925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.497426 (+ 501us) server_negotiation.cc:407] Connection header received
0619 22:23:29.497476 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.497479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.497541 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.497628 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:29.498546 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.499529 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.500718 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.501071 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.501162 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.503374 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:29.503378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:29.503381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:29.503414 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:29.505244 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:29.505744 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:29.505748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:29.505749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:29.505840 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:29.506114 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:29.506118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:29.506119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:29.506352 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:29.506473 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.507170 (+ 697us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.507314 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":113,"threads_started":1}
I20260619 22:23:29.511924 14099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:23:29.522270 14099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:23:29.546437 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.534453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37789 (local address 127.0.0.1:50038)
0619 22:23:29.537129 (+ 2676us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.537141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:29.537213 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:29.539549 (+ 2336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:29.539554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:29.539595 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:29.539755 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.539767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.542165 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.542169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.543570 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.543582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.544996 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.545000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.545124 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.545165 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 22:23:29.546097 (+ 932us) client_negotiation.cc:240] Negotiation successful
0619 22:23:29.546260 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2612,"thread_start_us":86,"threads_started":1}
I20260619 22:23:29.547134 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.535203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.1.74.131:37789)
0619 22:23:29.539053 (+ 3850us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.539061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.539087 (+ 26us) server_negotiation.cc:407] Connection header received
0619 22:23:29.539167 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.539173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.539389 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.539557 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:29.539907 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.541054 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.543744 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.544656 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.544769 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.544928 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.546877 (+ 1949us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.546991 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3736,"thread_start_us":130,"threads_started":1}
I20260619 22:23:29.549974 14506 tablet_service.cc:1467] Processing CreateTablet for tablet 65cc45ca8d0b46f8b2a78b3fda478d27 (DEFAULT_TABLE table=test-table [id=79d6b177cfba44519de067a26d67b2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:29.551168 14506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65cc45ca8d0b46f8b2a78b3fda478d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:29.562399 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.534335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42557 (local address 127.0.0.1:35190)
0619 22:23:29.536714 (+ 2379us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.536769 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:29.536900 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:29.556025 (+ 19125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:29.556029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:29.556052 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:29.556177 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.556184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.559568 (+ 3384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.559572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.560776 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.560788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.561913 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.561917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.562022 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.562052 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 22:23:29.562139 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 22:23:29.562235 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2168,"thread_start_us":102,"threads_started":1}
I20260619 22:23:29.563138 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.535183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.1.74.130:42557)
0619 22:23:29.539823 (+ 4640us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.539830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.539854 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:23:29.555678 (+ 15824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.555685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.555873 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.556345 (+ 472us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:29.556485 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.557665 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.560935 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.561763 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.561872 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.562013 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.562882 (+ 869us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.562992 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4510,"thread_start_us":84,"threads_started":1}
I20260619 22:23:29.564903 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.536027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:38807 (local address 127.0.0.1:50690)
0619 22:23:29.539030 (+ 3003us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.539045 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:29.539136 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:29.557832 (+ 18696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:29.557835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:29.557859 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:29.557950 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.557955 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.562098 (+ 4143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.562119 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.563345 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.563356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.564508 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.564511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.564614 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.564640 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:23:29.564687 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 22:23:29.564757 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2941,"thread_start_us":90,"threads_started":1}
I20260619 22:23:29.565775 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.536176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.1.74.129:38807)
0619 22:23:29.555091 (+ 18915us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.555098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.555134 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:23:29.555218 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.555223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.555449 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.558099 (+ 2650us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:29.558242 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.559420 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.563507 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.564346 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.565349 (+ 1003us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.565502 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.565552 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.565641 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18773,"thread_start_us":99,"threads_started":1}
I20260619 22:23:29.566437 14366 tablet_service.cc:1467] Processing CreateTablet for tablet 65cc45ca8d0b46f8b2a78b3fda478d27 (DEFAULT_TABLE table=test-table [id=79d6b177cfba44519de067a26d67b2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:29.567619 14366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65cc45ca8d0b46f8b2a78b3fda478d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:29.568858 14231 tablet_service.cc:1467] Processing CreateTablet for tablet 65cc45ca8d0b46f8b2a78b3fda478d27 (DEFAULT_TABLE table=test-table [id=79d6b177cfba44519de067a26d67b2f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:29.569845 14231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65cc45ca8d0b46f8b2a78b3fda478d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:29.571238 14598 tablet_bootstrap.cc:492] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Bootstrap starting.
I20260619 22:23:29.573661 14598 tablet_bootstrap.cc:654] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:29.579922 14598 log.cc:824] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:29.586421 14600 tablet_bootstrap.cc:492] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: Bootstrap starting.
I20260619 22:23:29.589534 14600 tablet_bootstrap.cc:654] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:29.590137 14599 tablet_bootstrap.cc:492] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Bootstrap starting.
I20260619 22:23:29.591835 14600 log.cc:824] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:29.593066 14598 tablet_bootstrap.cc:492] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: No bootstrap required, opened a new log
I20260619 22:23:29.593364 14598 ts_tablet_manager.cc:1397] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.000s
I20260619 22:23:29.597818 14598 raft_consensus.cc:348] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.598102 14598 raft_consensus.cc:374] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:29.598397 14600 tablet_bootstrap.cc:492] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: No bootstrap required, opened a new log
I20260619 22:23:29.598644 14600 ts_tablet_manager.cc:1397] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260619 22:23:29.598927 14598 raft_consensus.cc:729] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b26bfdbc65401989cebf3455dc2a05, State: Initialized, Role: FOLLOWER
I20260619 22:23:29.599370 14599 tablet_bootstrap.cc:654] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:29.599527 14598 consensus_queue.cc:260] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.600589 14599 log.cc:824] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:29.600694 14598 ts_tablet_manager.cc:1428] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 22:23:29.601131 14572 heartbeater.cc:503] Master 127.1.74.190:43275 was elected leader, sending a full tablet report...
I20260619 22:23:29.605401 14600 raft_consensus.cc:348] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.605744 14600 raft_consensus.cc:374] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:29.605868 14600 raft_consensus.cc:729] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 062e67a6109144048f7c7076071ee7ad, State: Initialized, Role: FOLLOWER
I20260619 22:23:29.606437 14600 consensus_queue.cc:260] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.607581 14297 heartbeater.cc:503] Master 127.1.74.190:43275 was elected leader, sending a full tablet report...
I20260619 22:23:29.608837 14599 tablet_bootstrap.cc:492] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: No bootstrap required, opened a new log
I20260619 22:23:29.609071 14599 ts_tablet_manager.cc:1397] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260619 22:23:29.611356 14600 ts_tablet_manager.cc:1428] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: Time spent starting tablet: real 0.013s user 0.008s sys 0.001s
I20260619 22:23:29.614373 14599 raft_consensus.cc:348] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.614714 14599 raft_consensus.cc:374] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:29.614773 14599 raft_consensus.cc:729] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8920704273948059c5f476668da14e6, State: Initialized, Role: FOLLOWER
I20260619 22:23:29.615392 14599 consensus_queue.cc:260] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.616151 14432 heartbeater.cc:503] Master 127.1.74.190:43275 was elected leader, sending a full tablet report...
I20260619 22:23:29.616603 14599 ts_tablet_manager.cc:1428] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260619 22:23:29.647877 14298 tablet.cc:2369] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:29.692960 14605 raft_consensus.cc:482] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:29.693213 14605 raft_consensus.cc:504] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.694635 14605 leader_election.cc:288] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8920704273948059c5f476668da14e6 (127.1.74.130:42557), d6b26bfdbc65401989cebf3455dc2a05 (127.1.74.131:37789)
I20260619 22:23:29.700429 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.695421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:39217 (local address 127.1.74.130:42557)
0619 22:23:29.695559 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.695564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.695631 (+ 67us) server_negotiation.cc:407] Connection header received
0619 22:23:29.695842 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.695846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.695897 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.695944 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:29.696383 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.697191 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.698977 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.699636 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.699741 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.699796 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.700190 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.700288 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 22:23:29.700492 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.695400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:48471 (local address 127.1.74.131:37789)
0619 22:23:29.695537 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.695543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.695855 (+ 312us) server_negotiation.cc:407] Connection header received
0619 22:23:29.696068 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.696071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.696125 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.696174 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:29.696611 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.697411 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.699022 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.699666 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.699769 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.699824 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.700257 (+ 433us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.700346 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:23:29.700917 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.695004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:37789 (local address 127.1.74.129:48471)
0619 22:23:29.695764 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.695779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:29.695936 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:29.696320 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:29.696323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:29.696340 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:29.696478 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.696485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.697588 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.697591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.698877 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.698888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.699928 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.699931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.700093 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.700116 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 22:23:29.700658 (+ 542us) client_negotiation.cc:240] Negotiation successful
0619 22:23:29.700746 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":81,"thread_start_us":103,"threads_started":1}
I20260619 22:23:29.701675 14517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65cc45ca8d0b46f8b2a78b3fda478d27" candidate_uuid: "062e67a6109144048f7c7076071ee7ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b26bfdbc65401989cebf3455dc2a05" is_pre_election: true
I20260619 22:23:29.702047 14517 raft_consensus.cc:2454] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 062e67a6109144048f7c7076071ee7ad in term 0.
I20260619 22:23:29.702409 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.694940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42557 (local address 127.1.74.129:39217)
0619 22:23:29.695536 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.695553 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:29.695715 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:29.696090 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:29.696093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:29.696109 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:29.696249 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.696255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.697344 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.697347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.698812 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:29.698823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.699901 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.699905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:29.700045 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.700069 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:23:29.701158 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0619 22:23:29.701241 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":42,"thread_start_us":155,"threads_started":1}
I20260619 22:23:29.702625 14185 leader_election.cc:302] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 062e67a6109144048f7c7076071ee7ad, d6b26bfdbc65401989cebf3455dc2a05; no voters:
I20260619 22:23:29.703008 14605 raft_consensus.cc:2781] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:23:29.703118 14605 raft_consensus.cc:482] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:23:29.703174 14605 raft_consensus.cc:3037] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:29.704576 14605 raft_consensus.cc:504] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.704984 14605 leader_election.cc:288] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [CANDIDATE]: Term 1 election: Requested vote from peers b8920704273948059c5f476668da14e6 (127.1.74.130:42557), d6b26bfdbc65401989cebf3455dc2a05 (127.1.74.131:37789)
I20260619 22:23:29.705427 14386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65cc45ca8d0b46f8b2a78b3fda478d27" candidate_uuid: "062e67a6109144048f7c7076071ee7ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8920704273948059c5f476668da14e6"
I20260619 22:23:29.705789 14386 raft_consensus.cc:3037] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:29.705912 14517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65cc45ca8d0b46f8b2a78b3fda478d27" candidate_uuid: "062e67a6109144048f7c7076071ee7ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b26bfdbc65401989cebf3455dc2a05"
I20260619 22:23:29.706051 14517 raft_consensus.cc:3037] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:29.707293 14386 raft_consensus.cc:2454] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 062e67a6109144048f7c7076071ee7ad in term 1.
I20260619 22:23:29.707806 14187 leader_election.cc:302] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 062e67a6109144048f7c7076071ee7ad, b8920704273948059c5f476668da14e6; no voters:
I20260619 22:23:29.705423 14380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65cc45ca8d0b46f8b2a78b3fda478d27" candidate_uuid: "062e67a6109144048f7c7076071ee7ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8920704273948059c5f476668da14e6" is_pre_election: true
I20260619 22:23:29.708109 14380 raft_consensus.cc:2362] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 062e67a6109144048f7c7076071ee7ad in term 1. Re-sending same reply.
W20260619 22:23:29.708163 14433 tablet.cc:2369] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:29.708547 14517 raft_consensus.cc:2454] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 062e67a6109144048f7c7076071ee7ad in term 1.
I20260619 22:23:29.709098 14605 raft_consensus.cc:2781] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:29.709353 14605 raft_consensus.cc:686] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [term 1 LEADER]: Becoming Leader. State: Replica: 062e67a6109144048f7c7076071ee7ad, State: Running, Role: LEADER
I20260619 22:23:29.709710 14605 consensus_queue.cc:237] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } }
I20260619 22:23:29.713379 14099 catalog_manager.cc:5495] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 062e67a6109144048f7c7076071ee7ad (127.1.74.129). New cstate: current_term: 1 leader_uuid: "062e67a6109144048f7c7076071ee7ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "062e67a6109144048f7c7076071ee7ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260619 22:23:29.731710 14597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.726936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.1.74.129:38807)
0619 22:23:29.727055 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:29.727059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:29.727075 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:23:29.727268 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:29.727274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:29.727349 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:29.727475 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:23:29.727751 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.728536 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.729827 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:29.730178 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:29.730300 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:29.730657 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:23:29.731260 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:23:29.731405 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:29.731480 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 22:23:29.731547 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 22:23:29.739178 14517 raft_consensus.cc:1261] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 1 FOLLOWER]: Refusing update from remote peer 062e67a6109144048f7c7076071ee7ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:29.740302 14609 consensus_queue.cc:1035] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b26bfdbc65401989cebf3455dc2a05" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:29.740854 14380 raft_consensus.cc:1261] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 1 FOLLOWER]: Refusing update from remote peer 062e67a6109144048f7c7076071ee7ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:29.741587 14609 consensus_queue.cc:1035] T 65cc45ca8d0b46f8b2a78b3fda478d27 P 062e67a6109144048f7c7076071ee7ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8920704273948059c5f476668da14e6" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 22:23:29.747476 14573 tablet.cc:2369] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:29.748414 14612 mvcc.cc:205] Tried to move back new op lower bound from 7298694388683325440 to 7298694388573921280. Current Snapshot: MvccSnapshot[applied={T|T < 7298694388683325440}]
I20260619 22:23:29.751238 14613 mvcc.cc:205] Tried to move back new op lower bound from 7298694388683325440 to 7298694388573921280. Current Snapshot: MvccSnapshot[applied={T|T < 7298694388683325440}]
I20260619 22:23:29.757413 14614 mvcc.cc:205] Tried to move back new op lower bound from 7298694388683325440 to 7298694388573921280. Current Snapshot: MvccSnapshot[applied={T|T < 7298694388683325440}]
I20260619 22:23:29.764569 14099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:29.764750 14099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:29.767697 14099 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23ce497ff98a4c38861a0711dfdbbeba: Sending DeleteTablet for 3 replicas of tablet 65cc45ca8d0b46f8b2a78b3fda478d27
I20260619 22:23:29.769008 14506 tablet_service.cc:1514] Processing DeleteTablet for tablet 65cc45ca8d0b46f8b2a78b3fda478d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:29 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:50038
I20260619 22:23:29.769555 14231 tablet_service.cc:1514] Processing DeleteTablet for tablet 65cc45ca8d0b46f8b2a78b3fda478d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:29 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:50690
I20260619 22:23:29.770632 14366 tablet_service.cc:1514] Processing DeleteTablet for tablet 65cc45ca8d0b46f8b2a78b3fda478d27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:29 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:35190
I20260619 22:23:29.771943 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14137
I20260619 22:23:29.772718 14621 tablet_replica.cc:330] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: stopping tablet replica
I20260619 22:23:29.773077 14621 raft_consensus.cc:2229] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:29.773455 14621 raft_consensus.cc:2258] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:29.775426 14621 ts_tablet_manager.cc:1905] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:29.778640 14621 ts_tablet_manager.cc:1918] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:29.779055 14623 tablet_replica.cc:330] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: stopping tablet replica
I20260619 22:23:29.779561 14623 raft_consensus.cc:2229] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:29.780046 14623 raft_consensus.cc:2258] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:29.780433 14621 log.cc:1196] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/65cc45ca8d0b46f8b2a78b3fda478d27
I20260619 22:23:29.782058 14623 ts_tablet_manager.cc:1905] T 65cc45ca8d0b46f8b2a78b3fda478d27 P b8920704273948059c5f476668da14e6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 22:23:29.783274 14085 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:38807 (error 108)
I20260619 22:23:29.783824 14621 ts_tablet_manager.cc:1939] T 65cc45ca8d0b46f8b2a78b3fda478d27 P d6b26bfdbc65401989cebf3455dc2a05: Deleting consensus metadata
I20260619 22:23:29.784216 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14301
I20260619 22:23:29.786186 14084 catalog_manager.cc:4841] TS d6b26bfdbc65401989cebf3455dc2a05 (127.1.74.131:37789): tablet 65cc45ca8d0b46f8b2a78b3fda478d27 (table test-table [id=79d6b177cfba44519de067a26d67b2f3]) successfully deleted
I20260619 22:23:29.788861 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:29.788514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:38807 (local address 127.0.0.1:50720)
0619 22:23:29.788636 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.788732 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:38807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 22:23:29.789072 14085 catalog_manager.cc:4584] TS 062e67a6109144048f7c7076071ee7ad (127.1.74.129:38807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65cc45ca8d0b46f8b2a78b3fda478d27: Network error: Client connection negotiation failed: client connection to 127.1.74.129:38807: connect: Connection refused (error 111)
W20260619 22:23:29.792510 14596 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:23:29.792091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42557 (local address 127.0.0.1:35192)
0619 22:23:29.792318 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:29.792395 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:42557: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":136}
I20260619 22:23:29.792879 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14435
I20260619 22:23:29.800571 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14066
I20260619 22:23:29.808813 1322 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 22:23:30.144855 1322 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 22:23:30.144994 1322 mini_ranger.cc:66] Stopping Ranger...
I20260619 22:23:30.491124 1322 mini_ranger.cc:68] Stopped Ranger
2026-06-19 22:23:30.491 UTC [13133] LOG: received smart shutdown request
2026-06-19 22:23:30.495 UTC [13133] LOG: background worker "logical replication launcher" (PID 13142) exited with exit code 1
2026-06-19 22:23:30.495 UTC [13137] LOG: shutting down
2026-06-19 22:23:30.672 UTC [13133] LOG: database system is shut down
2026-06-19T22:23:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55821 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:23:30 dist-test-slave-k84h krb5kdc[14633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:23:30 dist-test-slave-k84h krb5kdc[14633](info): set up 2 sockets
Jun 19 22:23:30 dist-test-slave-k84h krb5kdc[14633](info): commencing operation
krb5kdc: starting...
W20260619 22:23:31.851229 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:23:31 dist-test-slave-k84h krb5kdc[14633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:23:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:23:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:23:31.977666 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:43223
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:43223,127.1.74.189:36191,127.1.74.188:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:32.098824 14646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:32.099237 14646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:32.099327 14646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:32.104004 14646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:32.104128 14646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:32.104169 14646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:32.104209 14646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:32.104245 14646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:32.110124 14646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:43223,127.1.74.189:36191,127.1.74.188:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:43223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:32.111446 14646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:32.112576 14646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:32.121428 14651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:32.121457 14652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:32.121531 14654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:32.121683 14646 server_base.cc:1034] running on GCE node
I20260619 22:23:32.122345 14646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:32.123581 14646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:32.124801 14646 hybrid_clock.cc:648] HybridClock initialized: now 1781907812124785 us; error 24 us; skew 500 ppm
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:32.127755 14646 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:32.128695 14646 webserver.cc:460] Webserver started at http://127.1.74.190:42995/ using document root <none> and password file <none>
I20260619 22:23:32.129290 14646 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:32.129386 14646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:32.129626 14646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:32.131819 14646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "2dfd50e7cc744171844c6453e8467515"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "8bbf7e8e1517b0d8fee71370961a908a"
server_key_iv: "f4dd7605a772db2ead1d93a0af5417c1"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.132417 14646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "2dfd50e7cc744171844c6453e8467515"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "8bbf7e8e1517b0d8fee71370961a908a"
server_key_iv: "f4dd7605a772db2ead1d93a0af5417c1"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.136474 14646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:23:32.139163 14661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:32.140470 14646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 22:23:32.140623 14646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "2dfd50e7cc744171844c6453e8467515"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "8bbf7e8e1517b0d8fee71370961a908a"
server_key_iv: "f4dd7605a772db2ead1d93a0af5417c1"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.140967 14646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:32.158584 14646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:32.162122 14646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:32.162353 14646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:32.171860 14646 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:43223
I20260619 22:23:32.171883 14713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:43223 every 8 connection(s)
I20260619 22:23:32.173120 14646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:32.175379 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14646
I20260619 22:23:32.175554 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:23:32.175871 1322 external_mini_cluster.cc:1428] Setting key a19554a43f3d9af2d4cd395abc30baa0
I20260619 22:23:32.176708 14714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:32.184947 14714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.189@KRBTEST.COM" created.
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:32.202526 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.188857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43223 (local address 127.0.0.1:38988)
0619 22:23:32.189421 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.189467 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.190409 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.191277 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.191286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.191801 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.192202 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.192223 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.193474 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.193490 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.194228 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.194237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.194927 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.194931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.195029 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.195708 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.195725 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.197178 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.199262 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.199270 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.199280 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.199508 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.199923 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.199926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.199928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.200055 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.200721 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.200728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.200871 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.201061 (+ 190us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.201463 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":121,"spinlock_wait_cycles":14720,"thread_start_us":130,"threads_started":1}
I20260619 22:23:32.202534 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.189701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.1.74.190:43223)
0619 22:23:32.189963 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.189977 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.190024 (+ 47us) server_negotiation.cc:407] Connection header received
0619 22:23:32.190763 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.190771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.191057 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.191219 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.192430 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.193309 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.194362 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.194745 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.194939 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.197331 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.197337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.197348 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.197368 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.199086 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.199629 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.199639 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.199640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.199760 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.200156 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.200159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.200161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.200598 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.200741 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.201129 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.201414 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":160,"thread_start_us":129,"threads_started":1}
I20260619 22:23:32.205197 14714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab.
Entry for principal kudu/127.1.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab.
I20260619 22:23:32.206213 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.205855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51940)
0619 22:23:32.206008 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.206110 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 22:23:32.206321 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:36191
--webserver_interface=127.1.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab
--principal=kudu/127.1.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:43223,127.1.74.189:36191,127.1.74.188:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:32.206465 14663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
I20260619 22:23:32.207453 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.207184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51942)
0619 22:23:32.207312 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.207370 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 22:23:32.207597 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:36191: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
I20260619 22:23:32.256907 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } attempt: 1
I20260619 22:23:32.257869 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.257535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51956)
0619 22:23:32.257704 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.257783 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 22:23:32.258657 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.258391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51970)
0619 22:23:32.258514 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.258576 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 22:23:32.258886 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:36191: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
W20260619 22:23:32.324627 14721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:32.325014 14721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:32.325105 14721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 22:23:32.327050 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } attempt: 2
I20260619 22:23:32.327929 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.327595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51978)
0619 22:23:32.327763 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.327838 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 22:23:32.328658 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.328418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51982)
0619 22:23:32.328543 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.328596 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 22:23:32.328792 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:36191: Network error: Client connection negotiation failed: client connection to 127.1.74.189:36191: connect: Connection refused (error 111)
W20260619 22:23:32.330011 14721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:32.330065 14721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:32.330109 14721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:32.330147 14721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:32.330197 14721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:32.335863 14721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:43223,127.1.74.189:36191,127.1.74.188:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab
--principal=kudu/127.1.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:36191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:32.337162 14721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:32.338239 14721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:32.347958 14727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:32.347937 14729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:32.347937 14726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:32.348305 14721 server_base.cc:1034] running on GCE node
I20260619 22:23:32.348958 14721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:32.350170 14721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:32.351398 14721 hybrid_clock.cc:648] HybridClock initialized: now 1781907812351357 us; error 57 us; skew 500 ppm
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:32.355616 14721 init.cc:377] Logged in from keytab as kudu/127.1.74.189@KRBTEST.COM (short username kudu)
I20260619 22:23:32.356863 14721 webserver.cc:460] Webserver started at http://127.1.74.189:38323/ using document root <none> and password file <none>
I20260619 22:23:32.357539 14721 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:32.357646 14721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:32.357905 14721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:32.361011 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data/instance:
uuid: "b09adb6c2be64af090db618a51adeb3a"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "11722b18cbd36d7380e91be809b4fb3f"
server_key_iv: "f72153820c29911bac8cae8480b3ec7b"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.361785 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal/instance:
uuid: "b09adb6c2be64af090db618a51adeb3a"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "11722b18cbd36d7380e91be809b4fb3f"
server_key_iv: "f72153820c29911bac8cae8480b3ec7b"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.366138 14721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 22:23:32.369088 14736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:32.370810 14721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:23:32.371093 14721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal
uuid: "b09adb6c2be64af090db618a51adeb3a"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "11722b18cbd36d7380e91be809b4fb3f"
server_key_iv: "f72153820c29911bac8cae8480b3ec7b"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.371234 14721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:32.390568 14721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:32.395548 14721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:32.395890 14721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:32.404820 14721 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.189:36191
I20260619 22:23:32.404836 14788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.189:36191 every 8 connection(s)
I20260619 22:23:32.405933 14721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
I20260619 22:23:32.409024 14789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:32.413353 14789 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:23:32.414158 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14721
I20260619 22:23:32.414289 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-1/wal/instance
I20260619 22:23:32.414636 1322 external_mini_cluster.cc:1428] Setting key 3b580132e1f94759aac331c2239ed115
I20260619 22:23:32.419958 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } attempt: 3
WARNING: no policy specified for kudu/127.1.74.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.188@KRBTEST.COM" created.
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:23:32.433169 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.420474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51986)
0619 22:23:32.420668 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.420688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.420781 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.421378 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.421381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.421582 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.421682 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.421688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.423278 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.423283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.424240 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.424254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.424889 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.424892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.425105 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.425529 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.425540 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.428661 (+ 3121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.430988 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.430993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.430997 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.431315 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.431689 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.431695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.431700 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.431815 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.432592 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.432595 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.432695 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.432810 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.432963 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":4}
I20260619 22:23:32.434353 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.417168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43223 (local address 127.0.0.1:39002)
0619 22:23:32.417870 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.417922 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.418990 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.419613 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.419620 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.420002 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.420737 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.420754 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.422019 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.422022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.423236 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.423243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.423805 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.423809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.424029 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.424682 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.424698 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.428126 (+ 3428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.431243 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.431253 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.431266 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.431593 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.432111 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.432114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.432115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.432196 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.432722 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.432728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.432907 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.433058 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.433469 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":7,"thread_start_us":180,"threads_started":1}
I20260619 22:23:32.434374 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.420610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.1.74.189:36191)
0619 22:23:32.421002 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.421011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.421030 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:23:32.421061 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.421066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.421217 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.421394 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.421827 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.423148 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.424379 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.424760 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.424883 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.428854 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.428865 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.428884 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.428920 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.430815 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.431437 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.431444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.431446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.431578 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.431924 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.431926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.431927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.432483 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.432628 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.433052 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.433551 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":281,"thread_start_us":171,"threads_started":1}
I20260619 22:23:32.435104 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.416354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.1.74.190:43223)
0619 22:23:32.416565 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.416571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.418073 (+ 1502us) server_negotiation.cc:407] Connection header received
0619 22:23:32.419228 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.419232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.419305 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.419417 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.420961 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.421896 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.423355 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.423647 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.423766 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.428307 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.428312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.428316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.428351 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.431102 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.431936 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.431942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.431947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.432016 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.432311 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.432317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.432321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.432584 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.432731 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.434890 (+ 2159us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.435001 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":96}
I20260619 22:23:32.436182 14789 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:23:32.436715 14714 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:23:32.437376 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.437162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39718)
0619 22:23:32.437258 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.437310 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 22:23:32.438223 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.437990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39728)
0619 22:23:32.438120 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.438166 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260619 22:23:32.438398 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:23:32.445019 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.436750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:51996)
0619 22:23:32.436963 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.436977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.437079 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.437467 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.437471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.437654 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.437792 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.437799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.438766 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.438787 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.439437 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.439445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.439858 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.439860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.440001 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.440549 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.440562 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.441773 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.443604 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.443607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.443609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.443784 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.444054 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.444060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.444062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.444135 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.444505 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.444507 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.444588 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.444689 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.444847 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260619 22:23:32.445040 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.437109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.1.74.189:36191)
0619 22:23:32.437222 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.437228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.437241 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:23:32.437280 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.437283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.437335 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.437429 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.437915 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.438664 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.439533 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.439761 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.439841 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.441974 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.441976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.441978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.441998 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.443483 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.443887 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.443890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.443892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.443966 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.444227 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.444230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.444232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.444395 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.444505 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.444740 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.444915 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":37}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab.
Entry for principal kudu/127.1.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab.
I20260619 22:23:32.445895 14789 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:23:32.446600 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.446308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39734)
0619 22:23:32.446430 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.446521 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 22:23:32.446805 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:35097
--webserver_interface=127.1.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab
--principal=kudu/127.1.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:33709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:43223,127.1.74.189:36191,127.1.74.188:35097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:32.447299 14739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
I20260619 22:23:32.448145 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.447938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39748)
0619 22:23:32.448044 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.448088 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 22:23:32.448328 14789 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
I20260619 22:23:32.469565 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 1
I20260619 22:23:32.470435 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.470147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39752)
0619 22:23:32.470299 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.470357 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 22:23:32.471230 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.470986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39764)
0619 22:23:32.471108 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.471162 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 22:23:32.471398 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
I20260619 22:23:32.497651 14789 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 1
I20260619 22:23:32.498692 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.498359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39780)
0619 22:23:32.498520 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.498603 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 22:23:32.499485 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.499247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39790)
0619 22:23:32.499373 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.499428 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 22:23:32.499624 14789 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
I20260619 22:23:32.546577 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 2
I20260619 22:23:32.547457 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.547111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39800)
0619 22:23:32.547286 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.547362 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 22:23:32.548228 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.547976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39806)
0619 22:23:32.548103 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.548159 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 22:23:32.548424 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
W20260619 22:23:32.566313 14796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:32.566598 14796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:32.566639 14796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 22:23:32.567804 14789 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 2
I20260619 22:23:32.568715 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.568430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39810)
0619 22:23:32.568572 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.568640 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 22:23:32.569576 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.569320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39822)
0619 22:23:32.569458 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.569509 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 22:23:32.569763 14789 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
W20260619 22:23:32.571189 14796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:32.571245 14796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:32.571265 14796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:32.571282 14796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:32.571342 14796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:32.577180 14796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:43223,127.1.74.189:36191,127.1.74.188:35097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab
--principal=kudu/127.1.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:32.578377 14796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:32.579512 14796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:32.589948 14801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:32.589948 14802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:32.589951 14804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:32.590355 14796 server_base.cc:1034] running on GCE node
I20260619 22:23:32.591076 14796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:32.592247 14796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:32.593446 14796 hybrid_clock.cc:648] HybridClock initialized: now 1781907812593430 us; error 32 us; skew 500 ppm
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:32.596434 14796 init.cc:377] Logged in from keytab as kudu/127.1.74.188@KRBTEST.COM (short username kudu)
I20260619 22:23:32.597352 14796 webserver.cc:460] Webserver started at http://127.1.74.188:43057/ using document root <none> and password file <none>
I20260619 22:23:32.597882 14796 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:32.597944 14796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:32.598114 14796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:32.600246 14796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data/instance:
uuid: "d7dd849302e54d98b8e0145232ca2c89"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "3294f92af2e4168282385f0d6f679210"
server_key_iv: "95f4c23c039e558d98ce8c6e23f9c373"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.600795 14796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal/instance:
uuid: "d7dd849302e54d98b8e0145232ca2c89"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "3294f92af2e4168282385f0d6f679210"
server_key_iv: "95f4c23c039e558d98ce8c6e23f9c373"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.604532 14796 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:23:32.606803 14811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:32.608109 14796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:23:32.608224 14796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal
uuid: "d7dd849302e54d98b8e0145232ca2c89"
format_stamp: "Formatted at 2026-06-19 22:23:32 on dist-test-slave-k84h"
server_key: "3294f92af2e4168282385f0d6f679210"
server_key_iv: "95f4c23c039e558d98ce8c6e23f9c373"
server_key_version: "encryptionkey@0"
I20260619 22:23:32.608326 14796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:32.647616 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 3
I20260619 22:23:32.648494 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.648186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39832)
0619 22:23:32.648348 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.648417 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 22:23:32.649237 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.649017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39842)
0619 22:23:32.649128 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.649184 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 22:23:32.649389 14714 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
I20260619 22:23:32.654560 14796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:32.657832 14796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:32.658064 14796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:32.660943 14789 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 3
I20260619 22:23:32.661854 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.661453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39846)
0619 22:23:32.661677 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.661750 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260619 22:23:32.662700 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.662372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39850)
0619 22:23:32.662557 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.662619 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260619 22:23:32.662876 14789 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:35097: Network error: Client connection negotiation failed: client connection to 127.1.74.188:35097: connect: Connection refused (error 111)
I20260619 22:23:32.665688 14796 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.188:35097
I20260619 22:23:32.665709 14863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.188:35097 every 8 connection(s)
I20260619 22:23:32.666640 14796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
I20260619 22:23:32.669382 14864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:32.673120 14864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:23:32.674531 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14796
I20260619 22:23:32.674654 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781907643212257-1322-0/minicluster-data/master-2/wal/instance
I20260619 22:23:32.675045 1322 external_mini_cluster.cc:1428] Setting key 18bed300d8ce3ca8a8127527454db83a
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:23:32.687449 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.675181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.1.74.190:43223)
0619 22:23:32.675477 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.675484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.677991 (+ 2507us) server_negotiation.cc:407] Connection header received
0619 22:23:32.678126 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.678129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.678176 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.678274 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.679074 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.679569 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.680566 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.680797 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.680910 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.683359 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.683365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.683379 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.683414 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.685264 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.685813 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.685819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.685821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.685918 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.686272 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.686277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.686279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.686510 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.686627 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.687108 (+ 481us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.687253 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260619 22:23:32.688285 14868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.675703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:43223 (local address 127.0.0.1:39012)
0619 22:23:32.677858 (+ 2155us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.677893 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.678040 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.678320 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.678326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.678679 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.678933 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.678953 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.679693 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.679696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.680442 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.680451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.680954 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.680959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.681067 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.681565 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.681581 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.683192 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.685414 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.685423 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.685434 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.685689 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.686025 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.686030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.686032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.686162 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.686638 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.686648 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.686804 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.687047 (+ 243us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.687597 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2013,"mutex_wait_us":273,"spinlock_wait_cycles":7552,"thread_start_us":147,"threads_started":1}
I20260619 22:23:32.688287 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.675699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.1.74.188:35097)
0619 22:23:32.676142 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.676163 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.676196 (+ 33us) server_negotiation.cc:407] Connection header received
0619 22:23:32.676827 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.676842 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.677161 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.677398 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.678114 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.679213 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.680035 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.680589 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.680762 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.683766 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.683771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.683783 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.683804 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.685489 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.685942 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.685947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.685949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.686071 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.686352 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.686357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.686359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.686734 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.686965 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.687174 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.687362 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":269,"spinlock_wait_cycles":8064,"thread_start_us":134,"threads_started":1}
I20260619 22:23:32.689606 14864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:32.698901 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.690260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.1.74.189:36191)
0619 22:23:32.690471 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.690476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.690490 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:23:32.690536 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.690540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.690590 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.690666 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.691217 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.692016 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.693095 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.693451 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.693555 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.695607 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.695613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.695617 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.695651 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.697196 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.697658 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.697665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.697666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.697767 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.698046 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.698050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.698052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.698267 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.698387 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.698622 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.698734 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260619 22:23:32.698945 14868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.690085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:36191 (local address 127.0.0.1:52002)
0619 22:23:32.690214 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.690259 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.690348 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.690732 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.690738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.690952 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.691084 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.691093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.692133 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.692137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.692993 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.693001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.693581 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.693584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.693663 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.694139 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.694153 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.695469 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.697323 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.697329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.697332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.697535 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.697874 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.697876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.697877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.697954 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.698373 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.698375 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.698456 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.698539 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.698690 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 22:23:32.699745 14864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:23:32.700410 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.690967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.1.74.190:43223)
0619 22:23:32.691089 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.691093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.691108 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:23:32.691226 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.691228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.691263 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.691316 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.692026 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.692829 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.693865 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.694154 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.694262 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.696863 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.696867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.696879 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.696910 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.698516 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.699033 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.699036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.699037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.699126 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.699480 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.699483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.699485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.699705 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.699912 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.700123 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.700250 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:23:32.708940 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.700519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.1.74.188:35097)
0619 22:23:32.700628 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.700631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.700644 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:23:32.700674 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.700676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.700713 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.700792 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.701214 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.701793 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.702952 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.703346 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.703466 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.705782 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.705786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.705788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.705831 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.707297 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.707767 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.707773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.707775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.707857 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.708122 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.708127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.708129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.708296 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.708405 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.708640 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.708770 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":5888}
I20260619 22:23:32.708922 14868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.700296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39868)
0619 22:23:32.700404 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.700415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.700501 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.700835 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.700838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.701006 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.701096 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.701106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.701904 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.701907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.702774 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.702784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.703458 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.703461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.703560 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.704040 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.704053 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.705643 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.707422 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.707425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.707427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.707676 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.707954 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.707957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.707958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.708037 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.708394 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.708397 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.708478 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.708586 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.708720 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":20}
I20260619 22:23:32.712113 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.701782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.1.74.189:36191)
0619 22:23:32.701917 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.701921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.701940 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:23:32.702023 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.702025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.702056 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.702112 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.702906 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.703725 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.704790 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.705172 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.705294 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.708036 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.708039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.708042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.708064 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.710224 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.710713 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.710715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.710717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.710808 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.711168 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.711172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.711174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.711360 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.711603 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.711796 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.711946 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:23:32.713004 14864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89: Bootstrap starting.
I20260619 22:23:32.715536 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:23:32.715700 14864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:32.716441 14864 log.cc:824] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:32.718600 14864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89: No bootstrap required, opened a new log
I20260619 22:23:32.723026 14864 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.723325 14864 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:32.723395 14864 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7dd849302e54d98b8e0145232ca2c89, State: Initialized, Role: FOLLOWER
I20260619 22:23:32.723882 14864 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.724860 14877 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.725106 14877 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.725356 14864 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:32.727378 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.717839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.1.74.188:35097)
0619 22:23:32.717939 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.717945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.718336 (+ 391us) server_negotiation.cc:407] Connection header received
0619 22:23:32.718504 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.718507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.718547 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.718594 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.719648 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.720372 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.721492 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.721821 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.721903 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.723574 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.723578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.723580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.723609 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.725305 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.725805 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.725810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.725811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.725897 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.726245 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.726248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.726250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.726562 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.726660 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.727089 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.727212 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 22:23:32.727380 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.717797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.1.74.190:43223)
0619 22:23:32.717926 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.717930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.718169 (+ 239us) server_negotiation.cc:407] Connection header received
0619 22:23:32.718365 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.718369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.718422 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.718487 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.719644 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.720199 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.721088 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.721366 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.721494 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.723169 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.723174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.723177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.723222 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.725166 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.725735 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.725739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.725741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.725848 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.726163 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.726170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.726176 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.726419 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.726619 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.726936 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.727212 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 22:23:32.731460 14888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:23:32.731766 14888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:23:32.729239 14796 master_runner.cc:186] Error getting master registration for 127.1.74.190:43223: OK, instance_id { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" instance_seqno: 1781907812169228 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 22:23:32.732518 14796 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 22:23:32.735488 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.718752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.1.74.189:36191)
0619 22:23:32.718907 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.718913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.727329 (+ 8416us) server_negotiation.cc:407] Connection header received
0619 22:23:32.727378 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.727381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.727429 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.727505 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.728367 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.729065 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.730182 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.730435 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.730563 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.731977 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.731981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.731984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.732011 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.733651 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.734102 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.734108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.734110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.734201 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.734500 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.734506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.734508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.734705 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.734886 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.735170 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.735295 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:23:32.806095 14789 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 4
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:23:32.813527 14714 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } attempt: 4
I20260619 22:23:32.817273 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.806894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39886 (local address 127.1.74.188:35097)
0619 22:23:32.807015 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.807018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.807035 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:23:32.807156 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.807159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.807199 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.807280 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.807830 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.808470 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.809735 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.809970 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.810154 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.813737 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.813746 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.813753 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.813787 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.815471 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.815952 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.815956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.815958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.816044 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.816391 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.816395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.816397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.816600 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.816695 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.816993 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.817104 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 22:23:32.817294 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.806729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39886)
0619 22:23:32.806961 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.806976 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.807062 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.807326 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.807329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.807545 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.807656 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.807663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.808594 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.808598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.809592 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.809604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.810202 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.810206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.810309 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.811118 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.811140 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.813305 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.815586 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.815591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.815593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.815799 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.816148 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.816151 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.816153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.816284 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.816703 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.816706 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.816821 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.816926 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.817131 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
Jun 19 22:23:32 dist-test-slave-k84h krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907812, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:23:32.821056 14789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: Bootstrap starting.
I20260619 22:23:32.823521 14789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:32.823675 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.813925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:35097 (local address 127.0.0.1:39892)
0619 22:23:32.814137 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:32.814153 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:32.814231 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:32.814545 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:32.814551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:32.814734 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:32.814900 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.814908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.815957 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.815961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.816794 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:32.816803 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.817755 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.817758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:32.817825 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.818260 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:32.818276 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:32.819982 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:32.822053 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.822056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.822058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.822296 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.822603 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:32.822606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:32.822608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:32.822694 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:32.823185 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:32.823188 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:32.823288 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 22:23:32.823371 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 22:23:32.823514 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260619 22:23:32.824183 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:32.813982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.1.74.188:35097)
0619 22:23:32.814309 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:32.814318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:32.814331 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:23:32.814365 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:32.814368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:32.814408 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:32.814498 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:32.815051 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.815834 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.816907 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:32.817267 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:32.817438 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:32.820142 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:32.820146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:32.820149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:32.820173 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:32.821925 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.822420 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.822422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.822424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.822505 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:32.822803 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:32.822806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:32.822807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:32.823072 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:32.823764 (+ 692us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:32.823869 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 22:23:32.824004 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":104,"threads_started":1}
I20260619 22:23:32.824546 14789 log.cc:824] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:32.827535 14714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: Bootstrap starting.
I20260619 22:23:32.827723 14789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: No bootstrap required, opened a new log
I20260619 22:23:32.830325 14714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:32.831314 14714 log.cc:824] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:32.832453 14789 raft_consensus.cc:348] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.832825 14789 raft_consensus.cc:374] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:32.832963 14789 raft_consensus.cc:729] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09adb6c2be64af090db618a51adeb3a, State: Initialized, Role: FOLLOWER
I20260619 22:23:32.833554 14789 consensus_queue.cc:260] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.833782 14714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: No bootstrap required, opened a new log
I20260619 22:23:32.834193 14893 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.834421 14893 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.835016 14789 sys_catalog.cc:564] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:32.838357 14714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.838629 14714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:32.838718 14714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dfd50e7cc744171844c6453e8467515, State: Initialized, Role: FOLLOWER
I20260619 22:23:32.839246 14714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
W20260619 22:23:32.840258 14904 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:23:32.840395 14904 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:23:32.840605 14714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:32.838480 14721 master_runner.cc:186] Error getting master registration for 127.1.74.190:43223: OK, instance_id { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" instance_seqno: 1781907812169228 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 22:23:32.842562 14721 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 22:23:32.842820 14877 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:32.843030 14877 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.843945 14877 leader_election.cc:288] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dfd50e7cc744171844c6453e8467515 (127.1.74.190:43223), b09adb6c2be64af090db618a51adeb3a (127.1.74.189:36191)
I20260619 22:23:32.844331 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7dd849302e54d98b8e0145232ca2c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dfd50e7cc744171844c6453e8467515" is_pre_election: true
I20260619 22:23:32.844612 14688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7dd849302e54d98b8e0145232ca2c89 in term 0.
I20260619 22:23:32.844764 14763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7dd849302e54d98b8e0145232ca2c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09adb6c2be64af090db618a51adeb3a" is_pre_election: true
I20260619 22:23:32.845029 14763 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7dd849302e54d98b8e0145232ca2c89 in term 0.
I20260619 22:23:32.845181 14815 leader_election.cc:302] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dfd50e7cc744171844c6453e8467515, d7dd849302e54d98b8e0145232ca2c89; no voters:
I20260619 22:23:32.845460 14877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:23:32.845530 14877 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:23:32.845569 14877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:32.846906 14877 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
W20260619 22:23:32.847126 14917 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:23:32.847186 14917 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:23:32.847249 14877 leader_election.cc:288] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [CANDIDATE]: Term 1 election: Requested vote from peers 2dfd50e7cc744171844c6453e8467515 (127.1.74.190:43223), b09adb6c2be64af090db618a51adeb3a (127.1.74.189:36191)
I20260619 22:23:32.848023 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7dd849302e54d98b8e0145232ca2c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dfd50e7cc744171844c6453e8467515"
I20260619 22:23:32.847357 14906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.848163 14906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.848142 14688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:32.848467 14763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7dd849302e54d98b8e0145232ca2c89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b09adb6c2be64af090db618a51adeb3a"
I20260619 22:23:32.848575 14763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:32.849869 14688 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7dd849302e54d98b8e0145232ca2c89 in term 1.
I20260619 22:23:32.849869 14763 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7dd849302e54d98b8e0145232ca2c89 in term 1.
I20260619 22:23:32.850346 14813 leader_election.cc:302] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b09adb6c2be64af090db618a51adeb3a, d7dd849302e54d98b8e0145232ca2c89; no voters:
I20260619 22:23:32.852134 14877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:32.852444 14877 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [term 1 LEADER]: Becoming Leader. State: Replica: d7dd849302e54d98b8e0145232ca2c89, State: Running, Role: LEADER
I20260619 22:23:32.852893 14877 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } }
I20260619 22:23:32.855909 14646 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260619 22:23:32.856029 14646 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 22:23:32.856191 14877 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7dd849302e54d98b8e0145232ca2c89. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.856365 14877 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:32.856796 14921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:32.857772 14921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:32.863906 14763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [term 1 FOLLOWER]: Refusing update from remote peer d7dd849302e54d98b8e0145232ca2c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:32.864069 14688 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [term 1 FOLLOWER]: Refusing update from remote peer d7dd849302e54d98b8e0145232ca2c89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:32.864748 14877 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:32.864923 14919 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:32.868526 14893 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7dd849302e54d98b8e0145232ca2c89. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.868656 14893 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.870805 14906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7dd849302e54d98b8e0145232ca2c89. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.871518 14906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.873337 14919 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.873481 14919 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:32.873612 14922 mvcc.cc:205] Tried to move back new op lower bound from 7298694401483853824 to 7298694401448706048. Current Snapshot: MvccSnapshot[applied={T|T < 7298694401483853824}]
I20260619 22:23:32.874833 14906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.874972 14906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.875625 14924 mvcc.cc:205] Tried to move back new op lower bound from 7298694401483853824 to 7298694401448706048. Current Snapshot: MvccSnapshot[applied={T|T < 7298694401483853824}]
I20260619 22:23:32.875908 14921 catalog_manager.cc:1347] Generated new cluster ID: 046d843fd84642a297e4c223ad312866
I20260619 22:23:32.875978 14921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:32.888821 14877 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.889010 14877 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:32.890347 14893 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7dd849302e54d98b8e0145232ca2c89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfd50e7cc744171844c6453e8467515" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 43223 } } peers { permanent_uuid: "b09adb6c2be64af090db618a51adeb3a" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 36191 } } peers { permanent_uuid: "d7dd849302e54d98b8e0145232ca2c89" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 35097 } } }
I20260619 22:23:32.890470 14893 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:23:32.911957 14921 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:23:32.913465 14921 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:32.930649 14921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7dd849302e54d98b8e0145232ca2c89: Generated new TSK 0
I20260619 22:23:32.931658 14921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:23:33.841888 14904 catalog_manager.cc:1259] Loaded cluster ID: 046d843fd84642a297e4c223ad312866
I20260619 22:23:33.841974 14904 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: loading cluster ID for follower catalog manager: success
I20260619 22:23:33.843636 14904 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: acquiring CA information for follower catalog manager: success
I20260619 22:23:33.844913 14904 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b09adb6c2be64af090db618a51adeb3a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:23:33.848382 14917 catalog_manager.cc:1259] Loaded cluster ID: 046d843fd84642a297e4c223ad312866
I20260619 22:23:33.848490 14917 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: loading cluster ID for follower catalog manager: success
I20260619 22:23:33.850095 14917 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: acquiring CA information for follower catalog manager: success
I20260619 22:23:33.851306 14917 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2dfd50e7cc744171844c6453e8467515: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:23:34.006676 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14646
I20260619 22:23:34.013264 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14721
I20260619 22:23:34.019459 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14796
2026-06-19T22:23:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3229 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:23:34 dist-test-slave-k84h krb5kdc[14935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:23:34 dist-test-slave-k84h krb5kdc[14935](info): set up 2 sockets
Jun 19 22:23:34 dist-test-slave-k84h krb5kdc[14935](info): commencing operation
krb5kdc: starting...
W20260619 22:23:35.071081 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:23:35 dist-test-slave-k84h krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:23:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:23:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:23:35.201395 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40061
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:46021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:35.310796 14948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:35.311123 14948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:35.311167 14948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:35.315212 14948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:35.315274 14948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:35.315292 14948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:35.315310 14948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:35.315325 14948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:35.320994 14948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:46021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:35.322075 14948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:35.323081 14948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:35.331734 14954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:35.331722 14956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:35.331722 14953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:35.331957 14948 server_base.cc:1034] running on GCE node
I20260619 22:23:35.332687 14948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:35.333899 14948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:35.335085 14948 hybrid_clock.cc:648] HybridClock initialized: now 1781907815335068 us; error 35 us; skew 500 ppm
Jun 19 22:23:35 dist-test-slave-k84h krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907815, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:35.342450 14948 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:35.343641 14948 webserver.cc:460] Webserver started at http://127.1.74.190:41037/ using document root <none> and password file <none>
I20260619 22:23:35.344277 14948 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:35.344384 14948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:35.344659 14948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:35.346975 14948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "3fbf75a664c84bc281270d450e1a0c7f"
format_stamp: "Formatted at 2026-06-19 22:23:35 on dist-test-slave-k84h"
server_key: "e5f09c65dc0c67ca8d29c0b1b9f73432"
server_key_iv: "ef46cb0f899e3b4faea6d621b588ea32"
server_key_version: "encryptionkey@0"
I20260619 22:23:35.347620 14948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "3fbf75a664c84bc281270d450e1a0c7f"
format_stamp: "Formatted at 2026-06-19 22:23:35 on dist-test-slave-k84h"
server_key: "e5f09c65dc0c67ca8d29c0b1b9f73432"
server_key_iv: "ef46cb0f899e3b4faea6d621b588ea32"
server_key_version: "encryptionkey@0"
I20260619 22:23:35.351801 14948 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:23:35.354314 14963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:35.355697 14948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:23:35.355877 14948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "3fbf75a664c84bc281270d450e1a0c7f"
format_stamp: "Formatted at 2026-06-19 22:23:35 on dist-test-slave-k84h"
server_key: "e5f09c65dc0c67ca8d29c0b1b9f73432"
server_key_iv: "ef46cb0f899e3b4faea6d621b588ea32"
server_key_version: "encryptionkey@0"
I20260619 22:23:35.356030 14948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:35.380519 14948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:35.385079 14948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:35.385285 14948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:35.394992 14948 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40061
I20260619 22:23:35.395010 15015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40061 every 8 connection(s)
I20260619 22:23:35.396286 14948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:35.397899 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 14948
I20260619 22:23:35.398032 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:23:35.398351 1322 external_mini_cluster.cc:1428] Setting key cfdab64ff6264de0a703ea9b93dd1e18
I20260619 22:23:35.399861 15016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:35.406077 15016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f: Bootstrap starting.
I20260619 22:23:35.409192 15016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:35.410174 15016 log.cc:824] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:23:35 dist-test-slave-k84h krb5kdc[14935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:35.412711 15016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f: No bootstrap required, opened a new log
I20260619 22:23:35.417929 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:35.399711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.1.74.190:40061)
0619 22:23:35.400157 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:35.400171 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:35.400205 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:23:35.400812 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:35.400832 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:35.401133 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:35.401459 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:35.402219 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:35.403520 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:35.404468 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:35.405333 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:35.405507 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:35.413218 (+ 7711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:35.413223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:35.413236 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:35.413264 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:35.414986 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:35.415529 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:35.415535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:35.415541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:35.415656 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:35.415960 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:35.415963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:35.415965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:35.416370 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:35.416583 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:35.416845 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 22:23:35.417169 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":141,"threads_started":1}
I20260619 22:23:35.418567 15016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fbf75a664c84bc281270d450e1a0c7f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40061 } }
I20260619 22:23:35.418861 15016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:35.418922 15016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fbf75a664c84bc281270d450e1a0c7f, State: Initialized, Role: FOLLOWER
I20260619 22:23:35.419396 15016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fbf75a664c84bc281270d450e1a0c7f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40061 } }
I20260619 22:23:35.419550 15016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:35.419613 15016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:35.419706 15016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:35.420814 15016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fbf75a664c84bc281270d450e1a0c7f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40061 } }
I20260619 22:23:35.421177 15016 leader_election.cc:302] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fbf75a664c84bc281270d450e1a0c7f; no voters:
I20260619 22:23:35.421521 15016 leader_election.cc:288] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:23:35.421628 15021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:35.421847 15021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [term 1 LEADER]: Becoming Leader. State: Replica: 3fbf75a664c84bc281270d450e1a0c7f, State: Running, Role: LEADER
I20260619 22:23:35.422187 15021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fbf75a664c84bc281270d450e1a0c7f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40061 } }
I20260619 22:23:35.422821 15016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:35.423508 15021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fbf75a664c84bc281270d450e1a0c7f. Latest consensus state: current_term: 1 leader_uuid: "3fbf75a664c84bc281270d450e1a0c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fbf75a664c84bc281270d450e1a0c7f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40061 } } }
I20260619 22:23:35.423696 15021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:35.424084 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:23:35.424185 15029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:35.426306 15022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fbf75a664c84bc281270d450e1a0c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fbf75a664c84bc281270d450e1a0c7f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40061 } } }
I20260619 22:23:35.426406 15022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:35.427558 15029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:35.433149 15029 catalog_manager.cc:1347] Generated new cluster ID: ed049c1347df474c9b79664a36db15b4
I20260619 22:23:35.433228 15029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:35.471625 15029 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:23:35.472936 15029 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:35.484283 15029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fbf75a664c84bc281270d450e1a0c7f: Generated new TSK 0
I20260619 22:23:35.485179 15029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:23:35.531008 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 14948
2026-06-19T22:23:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1509 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:23:35 dist-test-slave-k84h krb5kdc[15043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:23:35 dist-test-slave-k84h krb5kdc[15043](info): set up 2 sockets
Jun 19 22:23:35 dist-test-slave-k84h krb5kdc[15043](info): commencing operation
krb5kdc: starting...
W20260619 22:23:36.607177 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:23:36 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:23:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:23:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:23:36.731000 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40885
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:36.840348 15056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:36.840623 15056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:36.840664 15056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:36.844606 15056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:36.844656 15056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:36.844676 15056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:36.844692 15056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:36.844707 15056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:36.849987 15056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:36.851106 15056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:36.852089 15056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:36.860826 15064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:36.860827 15062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:36.860867 15061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:36.861086 15056 server_base.cc:1034] running on GCE node
I20260619 22:23:36.861687 15056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:36.862958 15056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:36.864152 15056 hybrid_clock.cc:648] HybridClock initialized: now 1781907816864143 us; error 32 us; skew 500 ppm
Jun 19 22:23:36 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907816, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:36.872138 15056 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:36.873170 15056 webserver.cc:460] Webserver started at http://127.1.74.190:34519/ using document root <none> and password file <none>
I20260619 22:23:36.873785 15056 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:36.873883 15056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:36.874120 15056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:36.876372 15056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "b9ae399033d9477ba8e19c4ff45f1b26"
format_stamp: "Formatted at 2026-06-19 22:23:36 on dist-test-slave-k84h"
server_key: "89bb7fec15437efb1ba43f66886120d8"
server_key_iv: "2552b46d5010fc6a7dea6c8a7f78d786"
server_key_version: "encryptionkey@0"
I20260619 22:23:36.877004 15056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "b9ae399033d9477ba8e19c4ff45f1b26"
format_stamp: "Formatted at 2026-06-19 22:23:36 on dist-test-slave-k84h"
server_key: "89bb7fec15437efb1ba43f66886120d8"
server_key_iv: "2552b46d5010fc6a7dea6c8a7f78d786"
server_key_version: "encryptionkey@0"
I20260619 22:23:36.880982 15056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:23:36.883555 15071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:36.884846 15056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:23:36.885000 15056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "b9ae399033d9477ba8e19c4ff45f1b26"
format_stamp: "Formatted at 2026-06-19 22:23:36 on dist-test-slave-k84h"
server_key: "89bb7fec15437efb1ba43f66886120d8"
server_key_iv: "2552b46d5010fc6a7dea6c8a7f78d786"
server_key_version: "encryptionkey@0"
I20260619 22:23:36.885133 15056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:36.900899 15056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:36.904496 15056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:36.904678 15056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:36.912036 15056 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40885
I20260619 22:23:36.912336 15123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40885 every 8 connection(s)
I20260619 22:23:36.913122 15056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:36.916086 15124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:36.917510 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15056
I20260619 22:23:36.917647 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:23:36.917976 1322 external_mini_cluster.cc:1428] Setting key a39155c63f6954d1318e154ca24b0af2
I20260619 22:23:36.923061 15124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap starting.
I20260619 22:23:36.926011 15124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:36.927086 15124 log.cc:824] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:23:36 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:36.929189 15124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: No bootstrap required, opened a new log
I20260619 22:23:36.932940 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:36.919531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.1.74.190:40885)
0619 22:23:36.919953 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:36.919970 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:36.920007 (+ 37us) server_negotiation.cc:407] Connection header received
0619 22:23:36.920645 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:36.920663 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:36.920992 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:36.921268 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:36.922278 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:36.923377 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:36.924604 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:36.925059 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:36.925174 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:36.928328 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:36.928334 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:36.928348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:36.928373 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:36.930085 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:36.930538 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:36.930548 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:36.930553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:36.930660 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:36.930977 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:36.930981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:36.930983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:36.931373 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:36.931594 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:36.931870 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 22:23:36.932171 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":139,"threads_started":1}
I20260619 22:23:36.932878 15124 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:36.933199 15124 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:36.933265 15124 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ae399033d9477ba8e19c4ff45f1b26, State: Initialized, Role: FOLLOWER
I20260619 22:23:36.933705 15124 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:36.933872 15124 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:36.933936 15124 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:36.934038 15124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:36.936022 15124 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:36.936362 15124 leader_election.cc:302] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9ae399033d9477ba8e19c4ff45f1b26; no voters:
I20260619 22:23:36.936683 15124 leader_election.cc:288] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:23:36.936870 15129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:36.937129 15129 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 LEADER]: Becoming Leader. State: Replica: b9ae399033d9477ba8e19c4ff45f1b26, State: Running, Role: LEADER
I20260619 22:23:36.937428 15129 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:36.937992 15124 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:36.939163 15130 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } } }
I20260619 22:23:36.939294 15130 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:36.939688 15138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:36.939653 15131 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9ae399033d9477ba8e19c4ff45f1b26. Latest consensus state: current_term: 1 leader_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } } }
I20260619 22:23:36.939790 15131 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:36.942736 15138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:36.948602 15138 catalog_manager.cc:1347] Generated new cluster ID: 45ece4984c754c6e8a01fb81e3079920
I20260619 22:23:36.948697 15138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:36.965948 15138 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:23:36.966981 15138 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:36.976318 15138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Generated new TSK 0
I20260619 22:23:36.977051 15138 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:23:37.015057 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40885
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:37.126973 15150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:37.127236 15150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:37.127279 15150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:37.131394 15150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:37.131454 15150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:37.131553 15150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:23:37.136905 15150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:37.138101 15150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:37.139310 15150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:37.149001 15155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:37.149003 15158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:37.149127 15150 server_base.cc:1034] running on GCE node
W20260619 22:23:37.149036 15156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:37.149662 15150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:37.150354 15150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:37.151554 15150 hybrid_clock.cc:648] HybridClock initialized: now 1781907817151552 us; error 36 us; skew 500 ppm
Jun 19 22:23:37 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907817, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:37.154333 15150 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:23:37.155295 15150 webserver.cc:460] Webserver started at http://127.1.74.129:42335/ using document root <none> and password file <none>
I20260619 22:23:37.155848 15150 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:37.155992 15150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:37.156224 15150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:37.158200 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "3e86444c2d9b4072ad499221928628ad"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "d5932e78f56aab7bfe5ae85c5aab7a75"
server_key_iv: "70122a9fc7b52ecc3ee285c99596fd89"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.158751 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "3e86444c2d9b4072ad499221928628ad"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "d5932e78f56aab7bfe5ae85c5aab7a75"
server_key_iv: "70122a9fc7b52ecc3ee285c99596fd89"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.162429 15150 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:37.165014 15165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.166311 15150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:23:37.166466 15150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "3e86444c2d9b4072ad499221928628ad"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "d5932e78f56aab7bfe5ae85c5aab7a75"
server_key_iv: "70122a9fc7b52ecc3ee285c99596fd89"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.166590 15150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:37.179411 15150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:37.182610 15150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:37.182888 15150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:37.183517 15150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:37.184381 15150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:37.184453 15150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.184526 15150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:37.184600 15150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.194196 15150 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45457
I20260619 22:23:37.194221 15278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45457 every 8 connection(s)
I20260619 22:23:37.195209 15150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:23:37.201781 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15150
I20260619 22:23:37.201990 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:23:37.202332 1322 external_mini_cluster.cc:1428] Setting key ffb90452df408151d470c2767081505f
Jun 19 22:23:37 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907817, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:37.210328 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.197254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:39177 (local address 127.1.74.190:40885)
0619 22:23:37.197433 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.197437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.198348 (+ 911us) server_negotiation.cc:407] Connection header received
0619 22:23:37.199752 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.199756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.199815 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.199922 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:37.201146 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.201686 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.203123 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.203402 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.203516 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.206102 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:37.206106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:37.206109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:37.206142 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:37.207673 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.208447 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.208450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.208451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.208536 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.209023 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.209026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.209027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.209210 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:37.209287 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.210073 (+ 786us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.210198 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":78}
I20260619 22:23:37.211694 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.197668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:39177)
0619 22:23:37.198181 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.198226 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.199522 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.200103 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.200110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.200505 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:37.200994 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.201008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.201817 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.201825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.203000 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.203011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.203567 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.203571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.203777 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.204461 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:37.204477 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:37.205923 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:37.207845 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:37.207854 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:37.207876 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:37.208314 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:37.208650 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:37.208657 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:37.208662 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:37.208906 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:37.209328 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:37.209339 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:37.209757 (+ 418us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.210030 (+ 273us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.210454 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:23:37.213186 15279 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40885
I20260619 22:23:37.213464 15279 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:37.213997 15279 heartbeater.cc:511] Master 127.1.74.190:40885 requested a full tablet report, sending...
I20260619 22:23:37.215695 15088 ts_manager.cc:194] Registered new tserver with Master: 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:23:37.216873 15088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:39177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:23:37.232534 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40885
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:37.349077 15284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:37.349370 15284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:37.349414 15284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:37.353669 15284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:37.353753 15284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:37.353820 15284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:23:37.359558 15284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:37.360742 15284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:37.361814 15284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:37.372192 15292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:37.372225 15289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:37.372192 15290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:37.372659 15284 server_base.cc:1034] running on GCE node
I20260619 22:23:37.373136 15284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:37.373868 15284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:37.375078 15284 hybrid_clock.cc:648] HybridClock initialized: now 1781907817375056 us; error 44 us; skew 500 ppm
Jun 19 22:23:37 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907817, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:37.378096 15284 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:23:37.379115 15284 webserver.cc:460] Webserver started at http://127.1.74.130:39167/ using document root <none> and password file <none>
I20260619 22:23:37.379683 15284 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:37.379755 15284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:37.379938 15284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:37.382223 15284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "86faf5b30df54955b5a6b356fa66767a"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "c72fa7ad10d853480e967e17d5f3bf6a"
server_key_iv: "177fc43fa44d6f96c493c6af203317fa"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.382784 15284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "86faf5b30df54955b5a6b356fa66767a"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "c72fa7ad10d853480e967e17d5f3bf6a"
server_key_iv: "177fc43fa44d6f96c493c6af203317fa"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.386827 15284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:23:37.389787 15299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.391232 15284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 22:23:37.391340 15284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "86faf5b30df54955b5a6b356fa66767a"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "c72fa7ad10d853480e967e17d5f3bf6a"
server_key_iv: "177fc43fa44d6f96c493c6af203317fa"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.391444 15284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:37.402511 15284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:37.406155 15284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:37.406328 15284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:37.406965 15284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:37.407850 15284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:37.407907 15284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.407979 15284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:37.408010 15284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.417896 15284 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:43245
I20260619 22:23:37.417917 15412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:43245 every 8 connection(s)
I20260619 22:23:37.419009 15284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:23:37.419649 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15284
I20260619 22:23:37.419788 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:23:37.420169 1322 external_mini_cluster.cc:1428] Setting key ed058d873af2796224bc543dffd99540
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Jun 19 22:23:37 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907817, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:23:37.434257 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.421167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:33183 (local address 127.1.74.190:40885)
0619 22:23:37.421316 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.421320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.422675 (+ 1355us) server_negotiation.cc:407] Connection header received
0619 22:23:37.423904 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.423907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.423958 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.424046 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:37.425473 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.426019 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.427287 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.427516 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.427626 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.430301 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:37.430305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:37.430307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:37.430334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:37.431943 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.432603 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.432609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.432611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.432698 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.433103 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.433107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.433109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.433310 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:37.433417 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.433946 (+ 529us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.434081 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 22:23:37.435323 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.421706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:33183)
0619 22:23:37.422477 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.422539 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.423625 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.424267 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.424280 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.424813 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:37.425312 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.425324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.426154 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.426157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.427154 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.427164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.427672 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.427676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.427851 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.428582 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:37.428599 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:37.430102 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:37.432117 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:37.432123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:37.432145 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:37.432420 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:37.432793 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:37.432798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:37.432800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:37.432993 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:37.433416 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:37.433423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:37.433696 (+ 273us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.433905 (+ 209us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.434274 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":159,"threads_started":1}
I20260619 22:23:37.436635 15413 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40885
I20260619 22:23:37.436878 15413 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:37.437397 15413 heartbeater.cc:511] Master 127.1.74.190:40885 requested a full tablet report, sending...
I20260619 22:23:37.438499 15088 ts_manager.cc:194] Registered new tserver with Master: 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
I20260619 22:23:37.439301 15088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:33183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:23:37.450359 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40885
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:37.565646 15418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:37.566025 15418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:37.566074 15418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:37.570274 15418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:37.570338 15418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:37.570406 15418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:23:37.575971 15418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:37.577159 15418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:37.578166 15418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:37.588042 15423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:37.588205 15424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:37.588099 15426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:37.588598 15418 server_base.cc:1034] running on GCE node
I20260619 22:23:37.589039 15418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:37.589798 15418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:37.590961 15418 hybrid_clock.cc:648] HybridClock initialized: now 1781907817590930 us; error 43 us; skew 500 ppm
Jun 19 22:23:37 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907817, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:37.593868 15418 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:23:37.594905 15418 webserver.cc:460] Webserver started at http://127.1.74.131:41841/ using document root <none> and password file <none>
I20260619 22:23:37.595551 15418 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:37.595616 15418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:37.595846 15418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:37.598114 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "39969c35c2a9ed404f76fc6bec0ebcf2"
server_key_iv: "3daef9a0a906b41579bcf3a091590e93"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.598680 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "39969c35c2a9ed404f76fc6bec0ebcf2"
server_key_iv: "3daef9a0a906b41579bcf3a091590e93"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.602703 15418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 22:23:37.605463 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.606786 15418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:23:37.607002 15418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "39969c35c2a9ed404f76fc6bec0ebcf2"
server_key_iv: "3daef9a0a906b41579bcf3a091590e93"
server_key_version: "encryptionkey@0"
I20260619 22:23:37.607120 15418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:37.641620 15418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:37.645097 15418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:37.645314 15418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:37.645933 15418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:37.646780 15418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:37.646827 15418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.646970 15418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:37.647010 15418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:37.655546 15418 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:46213
I20260619 22:23:37.655632 15546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:46213 every 8 connection(s)
I20260619 22:23:37.656499 15418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:23:37.657743 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15418
I20260619 22:23:37.657902 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:23:37.658275 1322 external_mini_cluster.cc:1428] Setting key 13bcb61fe883c76a655cd641c62496d8
Jun 19 22:23:37 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907817, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:37.672194 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.658769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:34289 (local address 127.1.74.190:40885)
0619 22:23:37.658954 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.658960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.660288 (+ 1328us) server_negotiation.cc:407] Connection header received
0619 22:23:37.661398 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.661402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.661463 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.661552 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:37.662796 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.663428 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.664739 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.664991 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.665119 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.668000 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:37.668003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:37.668005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:37.668031 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:37.669567 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.670207 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.670210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.670211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.670297 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.670708 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.670711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.670712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.671073 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:37.671185 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.671859 (+ 674us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.672010 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 22:23:37.673215 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.659308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:34289)
0619 22:23:37.660143 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.660189 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.661158 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.661730 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.661738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.662136 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:37.662641 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.662655 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.663580 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.663584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.664596 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.664607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.665166 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.665171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.665371 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.665988 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:37.666006 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:37.667780 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:37.669694 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:37.669699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:37.669713 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:37.670084 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:37.670470 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:37.670472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:37.670474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:37.670621 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:37.671183 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:37.671190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:37.671564 (+ 374us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.671779 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.672200 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":127,"threads_started":1}
I20260619 22:23:37.674513 15547 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40885
I20260619 22:23:37.674741 15547 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:37.675268 15547 heartbeater.cc:511] Master 127.1.74.190:40885 requested a full tablet report, sending...
I20260619 22:23:37.676268 15088 ts_manager.cc:194] Registered new tserver with Master: 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213)
I20260619 22:23:37.676803 15088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:34289
I20260619 22:23:37.684847 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:23:37.694463 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.686838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.1.74.190:40885)
0619 22:23:37.687017 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.687023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.687232 (+ 209us) server_negotiation.cc:407] Connection header received
0619 22:23:37.687387 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.687390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.687441 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.687532 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:37.688244 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.688885 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.689733 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.689944 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.690057 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.691491 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:37.691495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:37.691498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:37.691530 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:37.692744 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.693168 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.693172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.693174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.693258 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:37.693537 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:37.693540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:37.693542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:37.693783 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:37.693935 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.694173 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.694288 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:23:37.697721 15088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:23:37.700763 15088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:23:37.715929 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.709358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56538 (local address 127.1.74.130:43245)
0619 22:23:37.710412 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.710430 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.710450 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:23:37.710531 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.710536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.710869 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.711152 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:37.711683 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.712569 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.714272 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.715140 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.715276 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.715449 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.715636 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.715742 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":140,"threads_started":1}
I20260619 22:23:37.715916 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.709141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:43245 (local address 127.0.0.1:56538)
0619 22:23:37.709635 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.709710 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.709840 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.711043 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.711046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.711080 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:37.711391 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.711404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.712774 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.712779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.714126 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.714136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.715273 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.715277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.715392 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.715434 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.715597 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.715743 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":79,"spinlock_wait_cycles":4736,"thread_start_us":102,"threads_started":1}
I20260619 22:23:37.716282 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.709360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45457 (local address 127.0.0.1:41930)
0619 22:23:37.710104 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.710117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.710196 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.711028 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.711034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.711079 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:37.711590 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.711599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.713264 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.713268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.714514 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.714526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.715655 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.715659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.715916 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.715978 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.716044 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.716143 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":406,"spinlock_wait_cycles":10240,"thread_start_us":118,"threads_started":1}
I20260619 22:23:37.716589 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.710467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.1.74.131:46213)
0619 22:23:37.710928 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.710942 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.710966 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:23:37.711040 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.711044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.711210 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.711392 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:37.711690 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.713104 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.714685 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.715490 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.716140 (+ 650us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.716278 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.716330 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.716439 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":153,"threads_started":1}
I20260619 22:23:37.716650 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.709868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46213 (local address 127.0.0.1:56288)
0619 22:23:37.710364 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.710375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.710451 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.711404 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.711407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.711423 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:37.711531 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.711538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.713264 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.713268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.714534 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.714543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.715657 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.715660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.715894 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.715980 (+ 86us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.716044 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.716520 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":655,"thread_start_us":111,"threads_started":1}
I20260619 22:23:37.717502 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.709952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.1.74.129:45457)
0619 22:23:37.710291 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.710309 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.710334 (+ 25us) server_negotiation.cc:407] Connection header received
0619 22:23:37.710466 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.710473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.710755 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.711349 (+ 594us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:37.711707 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.713104 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.714650 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.715490 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.717055 (+ 1565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.717213 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.717263 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.717353 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":102,"threads_started":1}
I20260619 22:23:37.719782 15213 tablet_service.cc:1467] Processing CreateTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 (DEFAULT_TABLE table=test-table [id=2126ddb106e94945b7d945698400e36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:37.719818 15481 tablet_service.cc:1467] Processing CreateTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 (DEFAULT_TABLE table=test-table [id=2126ddb106e94945b7d945698400e36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:37.719818 15347 tablet_service.cc:1467] Processing CreateTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 (DEFAULT_TABLE table=test-table [id=2126ddb106e94945b7d945698400e36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:37.720803 15213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ff44b1c9d34107a38ef4013ef91cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:37.720894 15481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ff44b1c9d34107a38ef4013ef91cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:37.720892 15347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85ff44b1c9d34107a38ef4013ef91cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:37.727387 15564 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Bootstrap starting.
I20260619 22:23:37.729123 15565 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Bootstrap starting.
I20260619 22:23:37.729473 15566 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Bootstrap starting.
I20260619 22:23:37.729831 15564 tablet_bootstrap.cc:654] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:37.730571 15564 log.cc:824] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:37.732131 15566 tablet_bootstrap.cc:654] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:37.732131 15565 tablet_bootstrap.cc:654] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:37.732662 15564 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: No bootstrap required, opened a new log
I20260619 22:23:37.732878 15564 ts_tablet_manager.cc:1397] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260619 22:23:37.733062 15565 log.cc:824] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:37.733062 15566 log.cc:824] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:37.735311 15566 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: No bootstrap required, opened a new log
I20260619 22:23:37.735535 15565 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: No bootstrap required, opened a new log
I20260619 22:23:37.735535 15566 ts_tablet_manager.cc:1397] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:23:37.735715 15565 ts_tablet_manager.cc:1397] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260619 22:23:37.736929 15564 raft_consensus.cc:348] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.737181 15564 raft_consensus.cc:374] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:37.737260 15564 raft_consensus.cc:729] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e86444c2d9b4072ad499221928628ad, State: Initialized, Role: FOLLOWER
I20260619 22:23:37.737717 15564 consensus_queue.cc:260] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.738641 15564 ts_tablet_manager.cc:1428] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260619 22:23:37.739100 15279 heartbeater.cc:503] Master 127.1.74.190:40885 was elected leader, sending a full tablet report...
I20260619 22:23:37.739845 15565 raft_consensus.cc:348] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.739832 15566 raft_consensus.cc:348] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.740128 15565 raft_consensus.cc:374] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:37.740186 15565 raft_consensus.cc:729] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86faf5b30df54955b5a6b356fa66767a, State: Initialized, Role: FOLLOWER
I20260619 22:23:37.740191 15566 raft_consensus.cc:374] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:37.740253 15566 raft_consensus.cc:729] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a72f93f7034a0b9fb94b7ff7ebf250, State: Initialized, Role: FOLLOWER
I20260619 22:23:37.740689 15565 consensus_queue.cc:260] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.740689 15566 consensus_queue.cc:260] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.741585 15547 heartbeater.cc:503] Master 127.1.74.190:40885 was elected leader, sending a full tablet report...
I20260619 22:23:37.741729 15565 ts_tablet_manager.cc:1428] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260619 22:23:37.741729 15566 ts_tablet_manager.cc:1428] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:23:37.742647 15413 heartbeater.cc:503] Master 127.1.74.190:40885 was elected leader, sending a full tablet report...
I20260619 22:23:37.845595 15570 raft_consensus.cc:482] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:37.845889 15570 raft_consensus.cc:504] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.847316 15570 leader_election.cc:288] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245), 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213)
I20260619 22:23:37.853240 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.847741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:46861 (local address 127.1.74.130:43245)
0619 22:23:37.847875 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.847881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.848216 (+ 335us) server_negotiation.cc:407] Connection header received
0619 22:23:37.848376 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.848379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.848435 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.848540 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:37.848937 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.849745 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.851811 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.852459 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.852599 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.852653 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.853023 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.853102 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:23:37.853240 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.847720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:54911 (local address 127.1.74.131:46213)
0619 22:23:37.847893 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.847905 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.848210 (+ 305us) server_negotiation.cc:407] Connection header received
0619 22:23:37.848370 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.848374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.848436 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.848534 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:37.848937 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.849738 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.851811 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.852459 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.852599 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.852653 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.853023 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.853102 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 22:23:37.853288 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.847592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46213 (local address 127.1.74.129:54911)
0619 22:23:37.848101 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.848117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.848243 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.848557 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.848560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.848587 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:37.848813 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.848822 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.849870 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.849873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.851645 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.851655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.852627 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.852631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.852848 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.852885 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.852979 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.853071 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":361,"spinlock_wait_cycles":30336,"thread_start_us":136,"threads_started":1}
I20260619 22:23:37.853291 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.847587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:43245 (local address 127.1.74.129:46861)
0619 22:23:37.848078 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.848093 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:37.848243 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:37.848557 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:37.848560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:37.848585 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:37.848816 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.848822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.849870 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.849873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.851641 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:37.851655 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.852627 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.852631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:37.852835 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.852894 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 22:23:37.853007 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 22:23:37.853071 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":337,"spinlock_wait_cycles":11520,"thread_start_us":143,"threads_started":1}
I20260619 22:23:37.854300 15501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ff44b1c9d34107a38ef4013ef91cb4" candidate_uuid: "3e86444c2d9b4072ad499221928628ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" is_pre_election: true
I20260619 22:23:37.854301 15367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ff44b1c9d34107a38ef4013ef91cb4" candidate_uuid: "3e86444c2d9b4072ad499221928628ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86faf5b30df54955b5a6b356fa66767a" is_pre_election: true
I20260619 22:23:37.854616 15501 raft_consensus.cc:2454] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e86444c2d9b4072ad499221928628ad in term 0.
I20260619 22:23:37.854627 15367 raft_consensus.cc:2454] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e86444c2d9b4072ad499221928628ad in term 0.
I20260619 22:23:37.855134 15168 leader_election.cc:302] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 3e86444c2d9b4072ad499221928628ad, 86faf5b30df54955b5a6b356fa66767a; no voters:
I20260619 22:23:37.855412 15570 raft_consensus.cc:2781] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:23:37.855494 15570 raft_consensus.cc:482] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:23:37.855537 15570 raft_consensus.cc:3037] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:37.856583 15570 raft_consensus.cc:504] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.856941 15570 leader_election.cc:288] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 1 election: Requested vote from peers 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245), 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213)
I20260619 22:23:37.857280 15367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ff44b1c9d34107a38ef4013ef91cb4" candidate_uuid: "3e86444c2d9b4072ad499221928628ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86faf5b30df54955b5a6b356fa66767a"
I20260619 22:23:37.857299 15501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ff44b1c9d34107a38ef4013ef91cb4" candidate_uuid: "3e86444c2d9b4072ad499221928628ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
I20260619 22:23:37.857429 15367 raft_consensus.cc:3037] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:37.857442 15501 raft_consensus.cc:3037] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:37.858572 15367 raft_consensus.cc:2454] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e86444c2d9b4072ad499221928628ad in term 1.
I20260619 22:23:37.858585 15501 raft_consensus.cc:2454] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e86444c2d9b4072ad499221928628ad in term 1.
I20260619 22:23:37.858902 15166 leader_election.cc:302] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e86444c2d9b4072ad499221928628ad, 86faf5b30df54955b5a6b356fa66767a; no voters:
I20260619 22:23:37.859117 15570 raft_consensus.cc:2781] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:37.859383 15570 raft_consensus.cc:686] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 LEADER]: Becoming Leader. State: Replica: 3e86444c2d9b4072ad499221928628ad, State: Running, Role: LEADER
I20260619 22:23:37.859754 15570 consensus_queue.cc:237] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:37.863042 15088 catalog_manager.cc:5495] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e86444c2d9b4072ad499221928628ad (127.1.74.129). New cstate: current_term: 1 leader_uuid: "3e86444c2d9b4072ad499221928628ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } health_report { overall_health: UNKNOWN } } }
I20260619 22:23:37.905030 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.901434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.1.74.129:45457)
0619 22:23:37.901593 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:37.901597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:37.901608 (+ 11us) server_negotiation.cc:407] Connection header received
0619 22:23:37.901682 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:37.901684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:37.901738 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:37.901855 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:23:37.902125 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.902630 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.903576 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:37.903792 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:37.903913 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:37.904215 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:23:37.904637 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:23:37.904770 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:37.904815 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 22:23:37.904882 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260619 22:23:37.907621 15548 tablet.cc:2369] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:37.910903 15367 raft_consensus.cc:1261] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Refusing update from remote peer 3e86444c2d9b4072ad499221928628ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:37.910919 15501 raft_consensus.cc:1261] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Refusing update from remote peer 3e86444c2d9b4072ad499221928628ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:37.911674 15570 consensus_queue.cc:1035] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:37.911939 15575 consensus_queue.cc:1035] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 22:23:37.920300 15414 tablet.cc:2369] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:37.920517 15579 mvcc.cc:205] Tried to move back new op lower bound from 7298694422155681792 to 7298694421956575232. Current Snapshot: MvccSnapshot[applied={T|T < 7298694422155681792}]
I20260619 22:23:37.922317 15580 mvcc.cc:205] Tried to move back new op lower bound from 7298694422155681792 to 7298694421956575232. Current Snapshot: MvccSnapshot[applied={T|T < 7298694422155681792}]
I20260619 22:23:37.929963 15084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:37.930145 15084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:37.932798 15084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Sending DeleteTablet for 3 replicas of tablet 85ff44b1c9d34107a38ef4013ef91cb4
I20260619 22:23:37.933736 15347 tablet_service.cc:1514] Processing DeleteTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:37 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56538
I20260619 22:23:37.933792 15213 tablet_service.cc:1514] Processing DeleteTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:37 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:41930
I20260619 22:23:37.933784 15481 tablet_service.cc:1514] Processing DeleteTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:37 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56288
I20260619 22:23:37.934348 15588 tablet_replica.cc:330] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: stopping tablet replica
I20260619 22:23:37.934540 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15150
I20260619 22:23:37.934712 15588 raft_consensus.cc:2229] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:23:37.935214 15590 tablet_replica.cc:330] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: stopping tablet replica
I20260619 22:23:37.935207 15589 tablet_replica.cc:330] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: stopping tablet replica
I20260619 22:23:37.935494 15590 raft_consensus.cc:2229] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:37.935493 15589 raft_consensus.cc:2229] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:37.935832 15590 raft_consensus.cc:2258] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:37.935832 15589 raft_consensus.cc:2258] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:37.937600 15590 ts_tablet_manager.cc:1905] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:37.937601 15589 ts_tablet_manager.cc:1905] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:37.940196 15590 ts_tablet_manager.cc:1918] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:37.940315 15590 log.cc:1196] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/85ff44b1c9d34107a38ef4013ef91cb4
I20260619 22:23:37.940660 15590 ts_tablet_manager.cc:1939] T 85ff44b1c9d34107a38ef4013ef91cb4 P 28a72f93f7034a0b9fb94b7ff7ebf250: Deleting consensus metadata
I20260619 22:23:37.941177 15589 ts_tablet_manager.cc:1918] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:37.941285 15589 log.cc:1196] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/85ff44b1c9d34107a38ef4013ef91cb4
I20260619 22:23:37.941597 15589 ts_tablet_manager.cc:1939] T 85ff44b1c9d34107a38ef4013ef91cb4 P 86faf5b30df54955b5a6b356fa66767a: Deleting consensus metadata
I20260619 22:23:37.942024 15074 catalog_manager.cc:4841] TS 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): tablet 85ff44b1c9d34107a38ef4013ef91cb4 (table test-table [id=2126ddb106e94945b7d945698400e36f]) successfully deleted
I20260619 22:23:37.942978 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15284
I20260619 22:23:37.942937 15072 catalog_manager.cc:4841] TS 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): tablet 85ff44b1c9d34107a38ef4013ef91cb4 (table test-table [id=2126ddb106e94945b7d945698400e36f]) successfully deleted
W20260619 22:23:37.943049 15074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:45457 (error 108)
I20260619 22:23:37.944702 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:37.944300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45457 (local address 127.0.0.1:41946)
0619 22:23:37.944483 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:37.944569 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260619 22:23:37.944904 15074 catalog_manager.cc:4584] TS 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85ff44b1c9d34107a38ef4013ef91cb4: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45457: connect: Connection refused (error 111)
I20260619 22:23:37.949203 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15418
I20260619 22:23:37.954593 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15056
I20260619 22:23:37.961205 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40885
--webserver_interface=127.1.74.190
--webserver_port=34519
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:38.072124 15592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:38.072449 15592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:38.072523 15592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:38.076746 15592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:38.076867 15592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:38.076910 15592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:38.076978 15592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:38.077019 15592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:38.082762 15592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:38.083962 15592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:38.085054 15592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:38.098795 15601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.098894 15599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.098795 15598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:38.099256 15592 server_base.cc:1034] running on GCE node
I20260619 22:23:38.099833 15592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:38.101023 15592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:38.102210 15592 hybrid_clock.cc:648] HybridClock initialized: now 1781907818102177 us; error 47 us; skew 500 ppm
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:38.105327 15592 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:38.106456 15592 webserver.cc:460] Webserver started at http://127.1.74.190:34519/ using document root <none> and password file <none>
I20260619 22:23:38.107208 15592 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:38.107295 15592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:38.111769 15592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:23:38.114080 15608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.115442 15592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:23:38.115581 15592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "b9ae399033d9477ba8e19c4ff45f1b26"
format_stamp: "Formatted at 2026-06-19 22:23:36 on dist-test-slave-k84h"
server_key: "89bb7fec15437efb1ba43f66886120d8"
server_key_iv: "2552b46d5010fc6a7dea6c8a7f78d786"
server_key_version: "encryptionkey@0"
I20260619 22:23:38.116508 15592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:38.146998 15592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:38.150399 15592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:38.150619 15592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:38.157822 15592 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40885
I20260619 22:23:38.158123 15660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40885 every 8 connection(s)
I20260619 22:23:38.159072 15592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:38.163466 15661 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:23:38.165181 15661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap starting.
I20260619 22:23:38.168515 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15592
I20260619 22:23:38.169003 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:45457
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:40885
--webserver_port=42335
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:23:38.183540 15661 log.cc:824] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:38.192759 15661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:23:38.193173 15661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap complete.
I20260619 22:23:38.197289 15661 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:38.197690 15661 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ae399033d9477ba8e19c4ff45f1b26, State: Initialized, Role: FOLLOWER
I20260619 22:23:38.198181 15661 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:38.198294 15661 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:38.198398 15661 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:38.198514 15661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:23:38.200565 15661 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:38.200910 15661 leader_election.cc:302] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9ae399033d9477ba8e19c4ff45f1b26; no voters:
I20260619 22:23:38.201251 15661 leader_election.cc:288] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 22:23:38.201364 15665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 22:23:38.201581 15665 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 LEADER]: Becoming Leader. State: Replica: b9ae399033d9477ba8e19c4ff45f1b26, State: Running, Role: LEADER
I20260619 22:23:38.201879 15665 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:38.202521 15661 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:38.202872 15665 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9ae399033d9477ba8e19c4ff45f1b26. Latest consensus state: current_term: 2 leader_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } } }
I20260619 22:23:38.203006 15665 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:38.203364 15669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:38.203529 15666 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } } }
I20260619 22:23:38.203794 15666 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:38.208310 15669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:38.209087 15669 catalog_manager.cc:1259] Loaded cluster ID: 45ece4984c754c6e8a01fb81e3079920
I20260619 22:23:38.209151 15669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:38.210562 15669 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 22:23:38.215492 15669 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 22:23:38.221324 15669 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:38.222206 15669 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Loaded TSK: 0
I20260619 22:23:38.223511 15669 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 22:23:38.288586 15663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:38.288940 15663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:38.289024 15663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:38.293365 15663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:38.293469 15663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:38.293566 15663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:23:38.299209 15663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:45457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=42335
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:38.300273 15663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:38.301295 15663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:38.312364 15689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.312366 15688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.312443 15691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:38.312830 15663 server_base.cc:1034] running on GCE node
I20260619 22:23:38.313265 15663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:38.313984 15663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:38.315155 15663 hybrid_clock.cc:648] HybridClock initialized: now 1781907818315142 us; error 34 us; skew 500 ppm
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:38.317989 15663 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:23:38.319084 15663 webserver.cc:460] Webserver started at http://127.1.74.129:42335/ using document root <none> and password file <none>
I20260619 22:23:38.319729 15663 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:38.319806 15663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:38.324023 15663 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:23:38.326282 15698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.327683 15663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:38.327781 15663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "3e86444c2d9b4072ad499221928628ad"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "d5932e78f56aab7bfe5ae85c5aab7a75"
server_key_iv: "70122a9fc7b52ecc3ee285c99596fd89"
server_key_version: "encryptionkey@0"
I20260619 22:23:38.328359 15663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:38.350919 15663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:38.354427 15663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:38.354651 15663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:38.355314 15663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:38.356474 15705 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 22:23:38.360530 15663 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 22:23:38.360612 15663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260619 22:23:38.360702 15663 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 22:23:38.362516 15663 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 22:23:38.362568 15663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 22:23:38.362998 15705 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Bootstrap starting.
I20260619 22:23:38.374318 15663 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45457
I20260619 22:23:38.374370 15812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45457 every 8 connection(s)
I20260619 22:23:38.375371 15663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:23:38.376513 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15663
I20260619 22:23:38.377112 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:43245
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:40885
--webserver_port=39167
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:23:38.381533 15705 log.cc:824] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:38.391784 15705 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:23:38.392215 15705 tablet_bootstrap.cc:492] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Bootstrap complete.
I20260619 22:23:38.393707 15705 ts_tablet_manager.cc:1397] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.008s
I20260619 22:23:38.393805 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.377776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:36335)
0619 22:23:38.379035 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.379081 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.379924 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.381071 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.381079 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.381527 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:38.382130 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.382155 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.383982 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.383985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.385018 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.385029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.385772 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.385776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.385974 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.386651 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:38.386667 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:38.388209 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:38.390467 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.390475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.390489 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.390832 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.391247 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.391251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.391254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.391390 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.392072 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:38.392077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:38.392323 (+ 246us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.392517 (+ 194us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.392856 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":153,"threads_started":1}
I20260619 22:23:38.393859 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.377836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:36335 (local address 127.1.74.190:40885)
0619 22:23:38.378388 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.378408 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.379213 (+ 805us) server_negotiation.cc:407] Connection header received
0619 22:23:38.380372 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.380383 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.380677 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.381019 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:38.382494 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.383803 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.385145 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.385620 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.385804 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.388422 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:38.388431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:38.388443 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:38.388476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:38.390303 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.391004 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.391009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.391015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.391146 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.391499 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.391502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.391504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.391883 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:38.392024 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.392652 (+ 628us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.392914 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":143,"threads_started":1}
I20260619 22:23:38.396492 15813 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40885
I20260619 22:23:38.396718 15813 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:38.396651 15705 raft_consensus.cc:348] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:38.397154 15705 raft_consensus.cc:729] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e86444c2d9b4072ad499221928628ad, State: Initialized, Role: FOLLOWER
I20260619 22:23:38.397214 15813 heartbeater.cc:511] Master 127.1.74.190:40885 requested a full tablet report, sending...
I20260619 22:23:38.397636 15705 consensus_queue.cc:260] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } } peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } }
I20260619 22:23:38.398749 15705 ts_tablet_manager.cc:1428] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260619 22:23:38.400030 15625 ts_manager.cc:194] Registered new tserver with Master: 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457)
I20260619 22:23:38.403992 15625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:36335
I20260619 22:23:38.406576 15813 heartbeater.cc:503] Master 127.1.74.190:40885 was elected leader, sending a full tablet report...
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:23:38.414436 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.402996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45457 (local address 127.0.0.1:41952)
0619 22:23:38.403374 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.403421 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.403553 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.405388 (+ 1835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.405394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.405740 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:38.405867 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.405882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.406945 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.406949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.407701 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.407710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.408420 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.408423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.408536 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.409237 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:38.409249 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:38.410654 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:38.412599 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.412608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.412614 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.412847 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.413176 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.413180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.413182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.413273 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.413743 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:38.413749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:38.413887 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.414036 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.414253 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":88,"threads_started":1}
I20260619 22:23:38.414450 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.403088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.1.74.129:45457)
0619 22:23:38.404807 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.404819 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.404839 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:23:38.404902 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.404907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.405139 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.405285 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:38.406004 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.406787 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.407824 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.408308 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.408407 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.410809 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:38.410819 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:38.410834 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:38.410909 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:38.412478 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.412971 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.412977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.412979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.413076 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.413373 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.413376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.413378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.413623 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:38.413770 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.414113 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.414255 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1626,"thread_start_us":111,"threads_started":1}
I20260619 22:23:38.415467 15747 tablet_service.cc:1514] Processing DeleteTablet for tablet 85ff44b1c9d34107a38ef4013ef91cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:37 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:41952
I20260619 22:23:38.416321 15822 tablet_replica.cc:330] stopping tablet replica
I20260619 22:23:38.416626 15822 raft_consensus.cc:2229] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:38.416877 15822 raft_consensus.cc:2258] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:38.417824 15822 ts_tablet_manager.cc:1905] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:38.421557 15822 ts_tablet_manager.cc:1918] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:38.421638 15822 log.cc:1196] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/85ff44b1c9d34107a38ef4013ef91cb4
I20260619 22:23:38.421921 15822 ts_tablet_manager.cc:1939] T 85ff44b1c9d34107a38ef4013ef91cb4 P 3e86444c2d9b4072ad499221928628ad: Deleting consensus metadata
I20260619 22:23:38.423643 15611 catalog_manager.cc:4841] TS 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457): tablet 85ff44b1c9d34107a38ef4013ef91cb4 (table test-table [id=2126ddb106e94945b7d945698400e36f]) successfully deleted
W20260619 22:23:38.500128 15817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:38.500435 15817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:38.500483 15817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:38.505035 15817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:38.505113 15817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:38.505179 15817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:23:38.511534 15817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:43245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=39167
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:38.512650 15817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:38.513634 15817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:38.523378 15827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.523481 15828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.523378 15830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:38.523644 15817 server_base.cc:1034] running on GCE node
I20260619 22:23:38.524155 15817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:38.524880 15817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:38.526098 15817 hybrid_clock.cc:648] HybridClock initialized: now 1781907818526064 us; error 48 us; skew 500 ppm
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:38.528908 15817 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:23:38.529819 15817 webserver.cc:460] Webserver started at http://127.1.74.130:39167/ using document root <none> and password file <none>
I20260619 22:23:38.530417 15817 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:38.530526 15817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:38.534646 15817 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:38.536811 15837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.538028 15817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:23:38.538165 15817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "86faf5b30df54955b5a6b356fa66767a"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "c72fa7ad10d853480e967e17d5f3bf6a"
server_key_iv: "177fc43fa44d6f96c493c6af203317fa"
server_key_version: "encryptionkey@0"
I20260619 22:23:38.538724 15817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:38.553614 15817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:38.556845 15817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:38.557090 15817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:38.557704 15817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:38.558578 15817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:38.558656 15817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.558729 15817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:38.558772 15817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.568137 15817 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:43245
I20260619 22:23:38.568157 15950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:43245 every 8 connection(s)
I20260619 22:23:38.569199 15817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:23:38.576922 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15817
I20260619 22:23:38.577415 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:46213
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:40885
--webserver_port=41841
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:38.586010 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.571090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:43395 (local address 127.1.74.190:40885)
0619 22:23:38.571233 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.571242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.572065 (+ 823us) server_negotiation.cc:407] Connection header received
0619 22:23:38.573390 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.573394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.573459 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.573571 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:38.574753 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.575364 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.576506 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.576745 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.576860 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.579734 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:38.579740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:38.579743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:38.579877 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:38.582033 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.582981 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.582986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.582990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.583101 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.583656 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.583662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.583665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.583980 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:38.584107 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.585638 (+ 1531us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.585800 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:23:38.586581 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.571448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:43395)
0619 22:23:38.571906 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.571950 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.573175 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.573753 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.573760 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.574126 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:38.574604 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.574617 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.575494 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.575497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.576367 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.576376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.576900 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.576904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.577104 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.577750 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:38.577767 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:38.579532 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:38.582254 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.582282 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.582305 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.582774 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.583244 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.583251 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.583255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.583517 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.584124 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:38.584151 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:38.584562 (+ 411us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.584835 (+ 273us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.585285 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260619 22:23:38.588163 15951 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40885
I20260619 22:23:38.588443 15951 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:38.589074 15951 heartbeater.cc:511] Master 127.1.74.190:40885 requested a full tablet report, sending...
I20260619 22:23:38.590497 15625 ts_manager.cc:194] Registered new tserver with Master: 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
I20260619 22:23:38.591275 15625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:43395
W20260619 22:23:38.696239 15954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:38.696508 15954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:38.696553 15954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:38.700567 15954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:38.700625 15954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:38.700686 15954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:23:38.706331 15954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:46213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=41841
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:38.707397 15954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:38.708388 15954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:38.718369 15962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:38.718370 15960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:38.718587 15954 server_base.cc:1034] running on GCE node
W20260619 22:23:38.718389 15959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:38.719275 15954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:38.720044 15954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:38.721233 15954 hybrid_clock.cc:648] HybridClock initialized: now 1781907818721192 us; error 54 us; skew 500 ppm
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:38.724013 15954 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:23:38.724952 15954 webserver.cc:460] Webserver started at http://127.1.74.131:41841/ using document root <none> and password file <none>
I20260619 22:23:38.725478 15954 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:38.725550 15954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:38.729701 15954 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260619 22:23:38.731823 15969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.733152 15954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:23:38.733254 15954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "39969c35c2a9ed404f76fc6bec0ebcf2"
server_key_iv: "3daef9a0a906b41579bcf3a091590e93"
server_key_version: "encryptionkey@0"
I20260619 22:23:38.733835 15954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:38.754240 15954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:38.757198 15954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:38.757356 15954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:38.757899 15954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:38.758674 15954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:38.758720 15954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.758761 15954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:38.758781 15954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:38.767845 15954 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:46213
I20260619 22:23:38.767868 16082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:46213 every 8 connection(s)
I20260619 22:23:38.768846 15954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:23:38.775609 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 15954
Jun 19 22:23:38 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907818, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:38.783418 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.770730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:51121 (local address 127.1.74.190:40885)
0619 22:23:38.770898 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.770901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.771969 (+ 1068us) server_negotiation.cc:407] Connection header received
0619 22:23:38.772837 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.772840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.772925 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.773004 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:38.774134 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.774643 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.776106 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.776449 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.776574 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.779319 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:38.779323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:38.779326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:38.779361 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:38.781059 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.781691 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.781699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.781700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.781777 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.782193 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.782196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.782197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.782362 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:38.782441 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.783114 (+ 673us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.783236 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:23:38.784435 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.776343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.1.74.190:40885)
0619 22:23:38.776565 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.776569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.776730 (+ 161us) server_negotiation.cc:407] Connection header received
0619 22:23:38.776890 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.776893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.776939 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.777030 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:38.777747 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.778338 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.779331 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.779544 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.779675 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.781185 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:38.781189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:38.781191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:38.781214 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:38.782764 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.783276 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.783280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.783282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.783352 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:38.783666 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:38.783669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:38.783671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:38.783864 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:38.784066 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.784193 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.784302 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260619 22:23:38.784441 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.771195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:51121)
0619 22:23:38.771844 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.771882 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.772627 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.773179 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.773185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.773533 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:38.773995 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.774007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.774745 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.774747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.775960 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.775973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.776680 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.776688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.776858 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.777528 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:38.777564 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:38.779155 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:38.781170 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.781176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.781192 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.781547 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.781870 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:38.781875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:38.781878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:38.782056 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:38.782469 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:38.782476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:38.782809 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.783021 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.783396 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":122,"threads_started":1}
I20260619 22:23:38.785825 16083 heartbeater.cc:347] Connected to a master server at 127.1.74.190:40885
I20260619 22:23:38.786162 16083 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:38.786717 16083 heartbeater.cc:511] Master 127.1.74.190:40885 requested a full tablet report, sending...
I20260619 22:23:38.787736 15625 ts_manager.cc:194] Registered new tserver with Master: 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213)
I20260619 22:23:38.788287 15625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:51121
I20260619 22:23:38.798489 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:23:38.803495 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.799255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.1.74.190:40885)
0619 22:23:38.799468 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.799475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.799645 (+ 170us) server_negotiation.cc:407] Connection header received
0619 22:23:38.799860 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.799863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.799990 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.800111 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:23:38.800365 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.800900 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.801847 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.802081 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.802195 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.802530 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:23:38.803084 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:23:38.803218 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.803277 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.803342 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260619 22:23:38.804219 15625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:23:38.807677 15625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:23:38.816356 15747 tablet_service.cc:1467] Processing CreateTablet for tablet 45026e1cab54487094c5a56bfdcadf1f (DEFAULT_TABLE table=test-table [id=8b6b4a0344624630b28c3433e62d0101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:38.817031 15747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45026e1cab54487094c5a56bfdcadf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:38.819975 15705 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Bootstrap starting.
I20260619 22:23:38.820400 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.815076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46213 (local address 127.0.0.1:56296)
0619 22:23:38.815254 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.815270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.815354 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.816191 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.816195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.816217 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:38.816327 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.816333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.817687 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.817691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.818984 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.818993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.819945 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.819949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.820084 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.820134 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.820190 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.820262 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":41}
I20260619 22:23:38.820969 15705 tablet_bootstrap.cc:654] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:38.821312 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.815249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.1.74.131:46213)
0619 22:23:38.815620 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.815637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.815661 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:23:38.815752 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.815761 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.816021 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.816190 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:38.816447 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.817536 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.819103 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.819794 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.820578 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.820727 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.820846 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.820969 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260619 22:23:38.821462 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.815428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:43245 (local address 127.0.0.1:56554)
0619 22:23:38.815714 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.815727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.815800 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.816803 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.816805 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.816819 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:38.816886 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.816890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.818199 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.818202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.819466 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.819478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.821018 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.821021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.821127 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.821157 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.821250 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.821318 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260619 22:23:38.821942 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.815765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.1.74.130:43245)
0619 22:23:38.816191 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.816203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.816223 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:23:38.816292 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.816298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.816477 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.816639 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:38.817014 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.817905 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.819642 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.820485 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.820597 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.820733 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.821678 (+ 945us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.821788 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
I20260619 22:23:38.823346 15705 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: No bootstrap required, opened a new log
I20260619 22:23:38.823470 15705 ts_tablet_manager.cc:1397] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 22:23:38.824064 15705 raft_consensus.cc:348] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.824219 15705 raft_consensus.cc:374] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:38.824272 15705 raft_consensus.cc:729] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e86444c2d9b4072ad499221928628ad, State: Initialized, Role: FOLLOWER
I20260619 22:23:38.824702 15705 consensus_queue.cc:260] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.825309 15705 ts_tablet_manager.cc:1428] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 22:23:38.825134 16017 tablet_service.cc:1467] Processing CreateTablet for tablet 45026e1cab54487094c5a56bfdcadf1f (DEFAULT_TABLE table=test-table [id=8b6b4a0344624630b28c3433e62d0101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:38.825181 15885 tablet_service.cc:1467] Processing CreateTablet for tablet 45026e1cab54487094c5a56bfdcadf1f (DEFAULT_TABLE table=test-table [id=8b6b4a0344624630b28c3433e62d0101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:38.826226 16017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45026e1cab54487094c5a56bfdcadf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:38.826227 15885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45026e1cab54487094c5a56bfdcadf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:38.833879 16096 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Bootstrap starting.
I20260619 22:23:38.835592 16097 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Bootstrap starting.
I20260619 22:23:38.837055 16096 tablet_bootstrap.cc:654] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:38.838572 16097 tablet_bootstrap.cc:654] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:38.838805 16096 log.cc:824] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:38.839573 16097 log.cc:824] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:38.841048 16096 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: No bootstrap required, opened a new log
I20260619 22:23:38.841274 16096 ts_tablet_manager.cc:1397] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260619 22:23:38.841850 16097 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: No bootstrap required, opened a new log
I20260619 22:23:38.842098 16097 ts_tablet_manager.cc:1397] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260619 22:23:38.844864 16096 raft_consensus.cc:348] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.845148 16096 raft_consensus.cc:374] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:38.845216 16096 raft_consensus.cc:729] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a72f93f7034a0b9fb94b7ff7ebf250, State: Initialized, Role: FOLLOWER
I20260619 22:23:38.845485 16097 raft_consensus.cc:348] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.845732 16097 raft_consensus.cc:374] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:38.845688 16096 consensus_queue.cc:260] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.845841 16097 raft_consensus.cc:729] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86faf5b30df54955b5a6b356fa66767a, State: Initialized, Role: FOLLOWER
I20260619 22:23:38.846375 16097 consensus_queue.cc:260] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.846457 16083 heartbeater.cc:503] Master 127.1.74.190:40885 was elected leader, sending a full tablet report...
I20260619 22:23:38.846764 16096 ts_tablet_manager.cc:1428] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 22:23:38.847189 15951 heartbeater.cc:503] Master 127.1.74.190:40885 was elected leader, sending a full tablet report...
I20260619 22:23:38.847499 16097 ts_tablet_manager.cc:1428] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260619 22:23:38.847468 16101 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:38.847972 16101 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.849887 16101 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457)
I20260619 22:23:38.854617 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.850906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:33793 (local address 127.1.74.131:46213)
0619 22:23:38.851006 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.851011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.851026 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:23:38.851071 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.851076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.851137 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.851213 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:38.851635 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.852330 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.853598 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.853974 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.854060 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.854101 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.854408 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.854490 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 22:23:38.854635 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.850116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46213 (local address 127.1.74.130:33793)
0619 22:23:38.850278 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.850295 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.850383 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.851334 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.851340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.851359 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:38.851473 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.851481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.852446 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.852452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.853445 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.853485 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.854094 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.854098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.854238 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.854278 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.854385 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.854469 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":52}
I20260619 22:23:38.854749 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.851050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:36713 (local address 127.1.74.129:45457)
0619 22:23:38.851223 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.851229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.851246 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:23:38.851306 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.851310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.851379 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.851501 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:38.851820 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.852446 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.853542 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.854164 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.854315 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.854486 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.854530 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.854601 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 22:23:38.854982 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.850361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45457 (local address 127.1.74.130:36713)
0619 22:23:38.850734 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.850747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:38.850825 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:38.851513 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:38.851517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:38.851537 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:38.851680 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.851686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.852563 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.852570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.853401 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:38.853410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.854294 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.854296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:38.854418 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.854440 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:23:38.854502 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 22:23:38.854581 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":17,"thread_start_us":134,"threads_started":1}
I20260619 22:23:38.855468 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45026e1cab54487094c5a56bfdcadf1f" candidate_uuid: "86faf5b30df54955b5a6b356fa66767a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" is_pre_election: true
I20260619 22:23:38.855558 15767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45026e1cab54487094c5a56bfdcadf1f" candidate_uuid: "86faf5b30df54955b5a6b356fa66767a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e86444c2d9b4072ad499221928628ad" is_pre_election: true
I20260619 22:23:38.855826 16037 raft_consensus.cc:2454] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86faf5b30df54955b5a6b356fa66767a in term 0.
I20260619 22:23:38.855854 15767 raft_consensus.cc:2454] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86faf5b30df54955b5a6b356fa66767a in term 0.
I20260619 22:23:38.856303 15840 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a; no voters:
I20260619 22:23:38.856616 16101 raft_consensus.cc:2781] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:23:38.856729 16101 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:23:38.856828 16101 raft_consensus.cc:3037] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:38.858026 16101 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.858374 16101 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [CANDIDATE]: Term 1 election: Requested vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457)
I20260619 22:23:38.858726 16037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45026e1cab54487094c5a56bfdcadf1f" candidate_uuid: "86faf5b30df54955b5a6b356fa66767a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
I20260619 22:23:38.858827 15767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45026e1cab54487094c5a56bfdcadf1f" candidate_uuid: "86faf5b30df54955b5a6b356fa66767a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e86444c2d9b4072ad499221928628ad"
I20260619 22:23:38.858899 16037 raft_consensus.cc:3037] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:38.859037 15767 raft_consensus.cc:3037] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:38.859973 16037 raft_consensus.cc:2454] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86faf5b30df54955b5a6b356fa66767a in term 1.
I20260619 22:23:38.860122 15767 raft_consensus.cc:2454] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86faf5b30df54955b5a6b356fa66767a in term 1.
I20260619 22:23:38.860253 15840 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a; no voters:
I20260619 22:23:38.860425 16101 raft_consensus.cc:2781] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:38.860674 16101 raft_consensus.cc:686] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 1 LEADER]: Becoming Leader. State: Replica: 86faf5b30df54955b5a6b356fa66767a, State: Running, Role: LEADER
I20260619 22:23:38.861009 16101 consensus_queue.cc:237] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:38.864513 15623 catalog_manager.cc:5495] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86faf5b30df54955b5a6b356fa66767a (127.1.74.130). New cstate: current_term: 1 leader_uuid: "86faf5b30df54955b5a6b356fa66767a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } health_report { overall_health: UNKNOWN } } }
W20260619 22:23:38.876713 15814 tablet.cc:2369] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:38.876916 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.873346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.1.74.130:43245)
0619 22:23:38.873509 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:38.873512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:38.873551 (+ 39us) server_negotiation.cc:407] Connection header received
0619 22:23:38.873630 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:38.873632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:38.873690 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:38.873777 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:23:38.874011 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.874546 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.875458 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:38.875670 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:38.875754 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:38.876023 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:23:38.876480 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:23:38.876575 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:38.876690 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 22:23:38.876743 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 22:23:38.883047 15767 raft_consensus.cc:1261] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Refusing update from remote peer 86faf5b30df54955b5a6b356fa66767a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:38.883049 16037 raft_consensus.cc:1261] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Refusing update from remote peer 86faf5b30df54955b5a6b356fa66767a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:38.883894 16101 consensus_queue.cc:1035] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:38.884140 16103 consensus_queue.cc:1035] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:38.893464 16107 mvcc.cc:205] Tried to move back new op lower bound from 7298694426136670208 to 7298694426057580544. Current Snapshot: MvccSnapshot[applied={T|T < 7298694426136670208}]
I20260619 22:23:38.893586 16108 mvcc.cc:205] Tried to move back new op lower bound from 7298694426136670208 to 7298694426057580544. Current Snapshot: MvccSnapshot[applied={T|T < 7298694426136670208 or (T in {7298694426136670208})}]
I20260619 22:23:38.900975 15625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:38.901165 15625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:38.903781 15625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Sending DeleteTablet for 3 replicas of tablet 45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:38.904824 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15663
I20260619 22:23:38.904976 16017 tablet_service.cc:1514] Processing DeleteTablet for tablet 45026e1cab54487094c5a56bfdcadf1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:38 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56296
I20260619 22:23:38.905483 15885 tablet_service.cc:1514] Processing DeleteTablet for tablet 45026e1cab54487094c5a56bfdcadf1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:38 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:56554
I20260619 22:23:38.906015 16118 tablet_replica.cc:330] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: stopping tablet replica
I20260619 22:23:38.906062 16117 tablet_replica.cc:330] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: stopping tablet replica
I20260619 22:23:38.906386 16118 raft_consensus.cc:2229] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:23:38.906400 16117 raft_consensus.cc:2229] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:38.906764 16117 raft_consensus.cc:2258] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:38.906778 16118 raft_consensus.cc:2258] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:38.908471 16117 ts_tablet_manager.cc:1905] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:38.908838 16118 ts_tablet_manager.cc:1905] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:38.911777 16118 ts_tablet_manager.cc:1918] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:38.912000 16118 log.cc:1196] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:38.912274 16117 ts_tablet_manager.cc:1918] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:38.912456 16118 ts_tablet_manager.cc:1939] T 45026e1cab54487094c5a56bfdcadf1f P 86faf5b30df54955b5a6b356fa66767a: Deleting consensus metadata
I20260619 22:23:38.912488 16117 log.cc:1196] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:38.912876 16117 ts_tablet_manager.cc:1939] T 45026e1cab54487094c5a56bfdcadf1f P 28a72f93f7034a0b9fb94b7ff7ebf250: Deleting consensus metadata
I20260619 22:23:38.913986 15609 catalog_manager.cc:4841] TS 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): tablet 45026e1cab54487094c5a56bfdcadf1f (table test-table [id=8b6b4a0344624630b28c3433e62d0101]) successfully deleted
I20260619 22:23:38.914374 15611 catalog_manager.cc:4841] TS 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): tablet 45026e1cab54487094c5a56bfdcadf1f (table test-table [id=8b6b4a0344624630b28c3433e62d0101]) successfully deleted
W20260619 22:23:38.914927 15611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:45457 (error 108)
I20260619 22:23:38.915146 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15817
I20260619 22:23:38.917769 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:38.917378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45457 (local address 127.0.0.1:41968)
0619 22:23:38.917557 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:38.917658 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260619 22:23:38.917996 15611 catalog_manager.cc:4584] TS 3e86444c2d9b4072ad499221928628ad (127.1.74.129:45457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45026e1cab54487094c5a56bfdcadf1f: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45457: connect: Connection refused (error 111)
I20260619 22:23:38.921720 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15954
I20260619 22:23:38.927251 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 15592
I20260619 22:23:38.933909 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40885
--webserver_interface=127.1.74.190
--webserver_port=34519
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:40885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:39.048908 16120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:39.049279 16120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:39.049362 16120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:39.053718 16120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:39.053851 16120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:39.053887 16120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:39.053932 16120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:39.053992 16120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:39.059718 16120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:40885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:40885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:39.060964 16120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:39.062045 16120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:39.070596 16125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:39.070585 16126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:39.070567 16128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:39.070878 16120 server_base.cc:1034] running on GCE node
I20260619 22:23:39.071544 16120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:39.072613 16120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:39.073776 16120 hybrid_clock.cc:648] HybridClock initialized: now 1781907819073759 us; error 30 us; skew 500 ppm
Jun 19 22:23:39 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:39.076480 16120 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:39.077404 16120 webserver.cc:460] Webserver started at http://127.1.74.190:34519/ using document root <none> and password file <none>
I20260619 22:23:39.077940 16120 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:39.078001 16120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:39.082093 16120 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:23:39.084430 16135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.085865 16120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:23:39.086028 16120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "b9ae399033d9477ba8e19c4ff45f1b26"
format_stamp: "Formatted at 2026-06-19 22:23:36 on dist-test-slave-k84h"
server_key: "89bb7fec15437efb1ba43f66886120d8"
server_key_iv: "2552b46d5010fc6a7dea6c8a7f78d786"
server_key_version: "encryptionkey@0"
I20260619 22:23:39.086624 16120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:39.109228 16120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:39.112767 16120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:39.113009 16120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:39.120023 16120 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:40885
I20260619 22:23:39.120348 16187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:40885 every 8 connection(s)
I20260619 22:23:39.121137 16120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:39.125365 16188 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:23:39.127039 16188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap starting.
I20260619 22:23:39.131093 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16120
I20260619 22:23:39.131569 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:45457
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:40885
--webserver_port=42335
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:23:39.145601 16188 log.cc:824] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:39.157073 16188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:23:39.157522 16188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: Bootstrap complete.
I20260619 22:23:39.161615 16188 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:39.162009 16188 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ae399033d9477ba8e19c4ff45f1b26, State: Initialized, Role: FOLLOWER
I20260619 22:23:39.162528 16188 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:39.162662 16188 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:39.162755 16188 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:39.162896 16188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 2 FOLLOWER]: Advancing to term 3
I20260619 22:23:39.164824 16188 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:39.165164 16188 leader_election.cc:302] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9ae399033d9477ba8e19c4ff45f1b26; no voters:
I20260619 22:23:39.165551 16188 leader_election.cc:288] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 22:23:39.165729 16192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 22:23:39.166074 16192 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [term 3 LEADER]: Becoming Leader. State: Replica: b9ae399033d9477ba8e19c4ff45f1b26, State: Running, Role: LEADER
I20260619 22:23:39.166415 16192 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } }
I20260619 22:23:39.166797 16188 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:39.168081 16194 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9ae399033d9477ba8e19c4ff45f1b26. Latest consensus state: current_term: 3 leader_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } } }
I20260619 22:23:39.168150 16193 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ae399033d9477ba8e19c4ff45f1b26" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 40885 } } }
I20260619 22:23:39.168263 16193 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:39.168213 16194 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26 [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:39.172042 16208 catalog_manager.cc:1259] Loaded cluster ID: 45ece4984c754c6e8a01fb81e3079920
I20260619 22:23:39.172125 16208 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: loading cluster ID for follower catalog manager: success
W20260619 22:23:39.172984 16208 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b9ae399033d9477ba8e19c4ff45f1b26: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260619 22:23:39.173072 16208 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260619 22:23:39.173185 16201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:39.175459 16201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:39.175976 16201 catalog_manager.cc:1259] Loaded cluster ID: 45ece4984c754c6e8a01fb81e3079920
I20260619 22:23:39.176045 16201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 22:23:39.176585 16201 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4aab0de9ad google::LogMessage::Fail() at ??:0
@ 0x7f4aab0e188a google::LogMessage::SendToLog() at ??:0
@ 0x7f4aab0de4de google::LogMessage::Flush() at ??:0
@ 0x7f4aab0dfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4ab5a21fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f4ab5a22d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4ab5a1f579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4ab5a63640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4aad3303d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4aad330c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f4aad3323f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f4aad321512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4aae9656db start_thread at ??:0
@ 0x7f4aa9bdc61f clone at ??:0
W20260619 22:23:39.266696 16190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:39.267087 16190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:39.267203 16190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:39.271420 16190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:39.271485 16190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:39.271548 16190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:23:39.277117 16190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:45457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=42335
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:39.278595 16190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:39.279678 16190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:39.290804 16229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:39.290784 16230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:39.290776 16232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:39.291473 16190 server_base.cc:1034] running on GCE node
I20260619 22:23:39.291880 16190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:39.292582 16190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:39.293802 16190 hybrid_clock.cc:648] HybridClock initialized: now 1781907819293781 us; error 69 us; skew 500 ppm
Jun 19 22:23:39 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:39.296970 16190 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:23:39.298018 16190 webserver.cc:460] Webserver started at http://127.1.74.129:42335/ using document root <none> and password file <none>
I20260619 22:23:39.298610 16190 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:39.298704 16190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:39.302819 16190 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:39.305325 16239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.306739 16190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 22:23:39.306921 16190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "3e86444c2d9b4072ad499221928628ad"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "d5932e78f56aab7bfe5ae85c5aab7a75"
server_key_iv: "70122a9fc7b52ecc3ee285c99596fd89"
server_key_version: "encryptionkey@0"
I20260619 22:23:39.307499 16190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:39.324744 16190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:39.328151 16190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:39.328367 16190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:39.329036 16190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:39.330283 16246 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 22:23:39.336110 16190 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 22:23:39.336215 16190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260619 22:23:39.336315 16190 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 22:23:39.338099 16190 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 22:23:39.338177 16190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 22:23:39.338510 16246 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Bootstrap starting.
I20260619 22:23:39.348985 16190 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45457
I20260619 22:23:39.349205 16353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45457 every 8 connection(s)
I20260619 22:23:39.350059 16190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:23:39.359263 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16190
I20260619 22:23:39.359760 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:43245
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:40885
--webserver_port=39167
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:23:39.361227 16246 log.cc:824] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:39.368911 16246 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:23:39.369336 16246 tablet_bootstrap.cc:492] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Bootstrap complete.
I20260619 22:23:39.370484 16246 ts_tablet_manager.cc:1397] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Time spent bootstrapping tablet: real 0.032s user 0.021s sys 0.007s
I20260619 22:23:39.373126 16246 raft_consensus.cc:348] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:39.373677 16246 raft_consensus.cc:729] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e86444c2d9b4072ad499221928628ad, State: Initialized, Role: FOLLOWER
I20260619 22:23:39.374188 16246 consensus_queue.cc:260] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:39.375278 16246 ts_tablet_manager.cc:1428] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260619 22:23:39.426213 16356 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:23:39.352082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:49571)
0619 22:23:39.352577 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.352632 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:39.353466 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:39.425424 (+ 71958us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":146,"threads_started":1}
W20260619 22:23:39.426832 16354 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:40885 (0 consecutive failures): Network error: Failed to ping master at 127.1.74.190:40885: Client connection negotiation failed: client connection to 127.1.74.190:40885: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 22:23:39.427591 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.427336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:56989)
0619 22:23:39.427461 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.427533 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 22:23:39.428274 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.428072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:40749)
0619 22:23:39.428172 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.428219 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 22:23:39.428467 16354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:23:39.492354 16357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:39.492614 16357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:39.492658 16357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:39.496893 16357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:39.496953 16357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:39.497046 16357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:23:39.502302 16357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:43245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=39167
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:39.503412 16357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:39.504423 16357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:39.514591 16365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:39.514591 16364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:39.514706 16357 server_base.cc:1034] running on GCE node
W20260619 22:23:39.514648 16367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:39.515327 16357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:39.515995 16357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:39.517185 16357 hybrid_clock.cc:648] HybridClock initialized: now 1781907819517150 us; error 49 us; skew 500 ppm
Jun 19 22:23:39 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:39.520063 16357 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:23:39.520982 16357 webserver.cc:460] Webserver started at http://127.1.74.130:39167/ using document root <none> and password file <none>
I20260619 22:23:39.521502 16357 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:39.521560 16357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:39.525439 16357 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:39.527522 16374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.528797 16357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:23:39.528893 16357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "86faf5b30df54955b5a6b356fa66767a"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "c72fa7ad10d853480e967e17d5f3bf6a"
server_key_iv: "177fc43fa44d6f96c493c6af203317fa"
server_key_version: "encryptionkey@0"
I20260619 22:23:39.529421 16357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:39.554996 16357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:39.558157 16357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:39.558318 16357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:39.558930 16357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:39.559724 16357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:39.559775 16357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.559818 16357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:39.559834 16357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.570910 16357 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:43245
I20260619 22:23:39.570982 16487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:43245 every 8 connection(s)
I20260619 22:23:39.572206 16357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:23:39.576750 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.574911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:55941)
0619 22:23:39.575756 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.575904 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":130,"threads_started":1}
W20260619 22:23:39.577245 16488 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:40885 (0 consecutive failures): Network error: Failed to ping master at 127.1.74.190:40885: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
I20260619 22:23:39.577867 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.577641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:38555)
0619 22:23:39.577768 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.577817 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 22:23:39.578047 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16357
I20260619 22:23:39.578552 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.578377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:44819)
0619 22:23:39.578467 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.578506 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 22:23:39.578545 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:46213
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:40885
--webserver_port=41841
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:39.578755 16488 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:23:39.693207 16491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:39.693516 16491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:39.693562 16491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:39.697522 16491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:39.697575 16491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:39.697630 16491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:23:39.702646 16491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:46213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=41841
--tserver_master_addrs=127.1.74.190:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:39.703634 16491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:39.704567 16491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:39.714349 16496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:39.714349 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:39.714362 16491 server_base.cc:1034] running on GCE node
W20260619 22:23:39.714499 16497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:39.715044 16491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:39.715706 16491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:39.716899 16491 hybrid_clock.cc:648] HybridClock initialized: now 1781907819716882 us; error 29 us; skew 500 ppm
Jun 19 22:23:39 dist-test-slave-k84h krb5kdc[15043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:39.719679 16491 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:23:39.720641 16491 webserver.cc:460] Webserver started at http://127.1.74.131:41841/ using document root <none> and password file <none>
I20260619 22:23:39.721206 16491 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:39.721273 16491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:39.725337 16491 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 22:23:39.727573 16506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.728892 16491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:23:39.729051 16491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "28a72f93f7034a0b9fb94b7ff7ebf250"
format_stamp: "Formatted at 2026-06-19 22:23:37 on dist-test-slave-k84h"
server_key: "39969c35c2a9ed404f76fc6bec0ebcf2"
server_key_iv: "3daef9a0a906b41579bcf3a091590e93"
server_key_version: "encryptionkey@0"
I20260619 22:23:39.729643 16491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:39.759590 16491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:39.762753 16491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:39.763007 16491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:39.763630 16491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:39.764467 16491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:39.764540 16491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.764636 16491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:39.764691 16491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:39.773248 16491 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:46213
I20260619 22:23:39.773267 16619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:46213 every 8 connection(s)
I20260619 22:23:39.774247 16491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:23:39.775645 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16491
I20260619 22:23:39.778080 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.776560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:57387)
0619 22:23:39.777268 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.777370 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":136,"threads_started":1}
W20260619 22:23:39.778596 16620 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:40885 (0 consecutive failures): Network error: Failed to ping master at 127.1.74.190:40885: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
I20260619 22:23:39.779187 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.778995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:58677)
0619 22:23:39.779092 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.779137 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 22:23:39.779762 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:39.779585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:43501)
0619 22:23:39.779677 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:39.779716 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 22:23:39.779978 16620 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 22:23:40.429622 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:40.429153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:48515)
0619 22:23:40.429425 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:40.429496 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260619 22:23:40.579908 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:40.579407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:49721)
0619 22:23:40.579719 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:40.579787 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":91,"threads_started":1}
I20260619 22:23:40.781155 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:40.780593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:46377)
0619 22:23:40.780940 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:40.781012 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
I20260619 22:23:41.088362 16627 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:41.088634 16627 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:41.089845 16627 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
Jun 19 22:23:41 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.130@KRBTEST.COM
Jun 19 22:23:41 dist-test-slave-k84h krb5kdc[15043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907819, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.131@KRBTEST.COM
I20260619 22:23:41.105710 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.090085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46213 (local address 127.1.74.129:42827)
0619 22:23:41.090425 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:41.090442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:41.090560 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:41.092494 (+ 1934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:41.092500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:41.093088 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:41.093620 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:41.093639 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.095093 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.095096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:41.096166 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:41.096177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.096954 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.096958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:41.097198 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:41.098124 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:41.098150 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:41.100283 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:41.103043 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:41.103047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:41.103051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:41.103296 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:41.103698 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:41.103706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:41.103707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:41.103882 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:41.104597 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:41.104604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:41.104906 (+ 302us) client_negotiation.cc:769] Sending connection context
0619 22:23:41.105112 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 22:23:41.105510 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":288,"spinlock_wait_cycles":62336,"thread_start_us":79,"threads_started":1}
I20260619 22:23:41.105723 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.090246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:42827 (local address 127.1.74.131:46213)
0619 22:23:41.090793 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:41.090812 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:41.090892 (+ 80us) server_negotiation.cc:407] Connection header received
0619 22:23:41.091755 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:41.091762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:41.092081 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:41.092383 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:41.093828 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.094915 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.096335 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.096809 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.097074 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:41.100504 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:41.100510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:41.100522 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:41.100557 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:41.102792 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:41.103429 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:41.103436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:41.103448 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:41.103586 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:41.103988 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:41.103994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:41.103997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:41.104457 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:41.104611 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:41.105257 (+ 646us) server_negotiation.cc:299] Negotiation successful
0619 22:23:41.105530 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":63,"threads_started":1}
I20260619 22:23:41.105760 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.090343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:42981 (local address 127.1.74.130:43245)
0619 22:23:41.090693 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:41.090709 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:41.090753 (+ 44us) server_negotiation.cc:407] Connection header received
0619 22:23:41.091692 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:41.091700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:41.092081 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:41.092417 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:41.093853 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.095331 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.096316 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.096806 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.097073 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:41.100287 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:41.100296 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:41.100316 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:41.100363 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:41.102659 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:41.103450 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:41.103456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:41.103462 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:41.103605 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:41.104012 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:41.104016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:41.104017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:41.104469 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:41.104619 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:41.105239 (+ 620us) server_negotiation.cc:299] Negotiation successful
0619 22:23:41.105560 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260619 22:23:41.105758 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.090197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:43245 (local address 127.1.74.129:42981)
0619 22:23:41.090526 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:41.090542 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:41.090654 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:41.092494 (+ 1840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:41.092500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:41.093087 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:41.093621 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:41.093638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.095506 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.095509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:41.096168 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:41.096178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:41.096954 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:41.096958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:41.097194 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:41.098147 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:41.098158 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:41.099956 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:41.102786 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:41.102794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:41.102809 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:41.103300 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:41.103717 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:41.103721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:41.103723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:41.103892 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:41.104597 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:41.104604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:41.104905 (+ 301us) client_negotiation.cc:769] Sending connection context
0619 22:23:41.105111 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 22:23:41.105598 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":307,"spinlock_wait_cycles":17664,"thread_start_us":89,"threads_started":1}
W20260619 22:23:41.108981 16242 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
W20260619 22:23:41.109453 16240 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:41.109539 16240 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e86444c2d9b4072ad499221928628ad; no voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a
I20260619 22:23:41.109862 16627 raft_consensus.cc:2726] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 22:23:41.430758 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.430463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:33683)
0619 22:23:41.430609 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:41.430671 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 22:23:41.581346 16632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.580860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:41285)
0619 22:23:41.581158 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:41.581219 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":106,"threads_started":1}
I20260619 22:23:41.782687 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:41.782224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:41305)
0619 22:23:41.782515 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:41.782583 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260619 22:23:42.432256 16634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:42.431694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:43929)
0619 22:23:42.432014 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:42.432082 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260619 22:23:42.583127 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:42.582483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:35125)
0619 22:23:42.582887 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:42.582974 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":108,"threads_started":1}
I20260619 22:23:42.784008 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:42.783515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:54591)
0619 22:23:42.783807 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:42.783868 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":76,"threads_started":1}
I20260619 22:23:43.181836 16637 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:43.182016 16637 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:43.182403 16637 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
W20260619 22:23:43.183331 16240 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
W20260619 22:23:43.183426 16242 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:43.183517 16242 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e86444c2d9b4072ad499221928628ad; no voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a
I20260619 22:23:43.183657 16637 raft_consensus.cc:2726] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 22:23:43.433736 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:43.433228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:54299)
0619 22:23:43.433551 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:43.433623 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260619 22:23:43.584466 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:43.583984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:34137)
0619 22:23:43.584294 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:43.584364 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
I20260619 22:23:43.785377 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:43.784893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:48513)
0619 22:23:43.785219 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:43.785281 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":127,"threads_started":1}
I20260619 22:23:44.435082 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:44.434609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:50281)
0619 22:23:44.434924 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:44.434979 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":67,"threads_started":1}
I20260619 22:23:44.586048 16642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:44.585452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:38021)
0619 22:23:44.585816 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:44.585886 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":82,"threads_started":1}
I20260619 22:23:44.786602 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:44.786146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:47659)
0619 22:23:44.786413 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:44.786506 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":89,"threads_started":1}
I20260619 22:23:45.436420 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:45.435831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:60185)
0619 22:23:45.436185 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:45.436272 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
I20260619 22:23:45.587445 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:45.586969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:56999)
0619 22:23:45.587280 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:45.587345 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260619 22:23:45.787910 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:45.787429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:44217)
0619 22:23:45.787691 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:45.787757 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
I20260619 22:23:45.892052 16647 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:45.892235 16647 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:45.892721 16647 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
W20260619 22:23:45.893522 16240 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
W20260619 22:23:45.893682 16242 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:45.893775 16242 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e86444c2d9b4072ad499221928628ad; no voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a
I20260619 22:23:45.893935 16647 raft_consensus.cc:2726] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 22:23:46.437863 16648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:46.437242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:56087)
0619 22:23:46.437655 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:46.437731 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":79,"threads_started":1}
I20260619 22:23:46.589028 16649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:46.588419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:59849)
0619 22:23:46.588793 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:46.588885 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":96,"threads_started":1}
I20260619 22:23:46.789536 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:46.789027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:41923)
0619 22:23:46.789372 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:46.789437 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":90,"threads_started":1}
I20260619 22:23:47.439045 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:47.438585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:48331)
0619 22:23:47.438828 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:47.438949 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260619 22:23:47.590348 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:47.589887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:51969)
0619 22:23:47.590182 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:47.590247 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260619 22:23:47.790942 16653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:47.790418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:53357)
0619 22:23:47.790704 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:47.790780 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
I20260619 22:23:48.440183 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:48.439757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:60671)
0619 22:23:48.440022 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:48.440084 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
I20260619 22:23:48.591595 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:48.591178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:60121)
0619 22:23:48.591435 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:48.591496 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260619 22:23:48.792315 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:48.791843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:58891)
0619 22:23:48.792154 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:48.792216 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":100,"threads_started":1}
I20260619 22:23:48.894232 16657 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:48.894431 16657 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:48.895025 16657 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
W20260619 22:23:48.895745 16242 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
W20260619 22:23:48.895891 16240 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:48.895983 16240 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e86444c2d9b4072ad499221928628ad; no voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a
I20260619 22:23:48.896142 16657 raft_consensus.cc:2726] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 22:23:49.441710 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:49.441205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:43413)
0619 22:23:49.441539 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:49.441606 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260619 22:23:49.592928 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:49.592490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:40759)
0619 22:23:49.592769 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:49.592827 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":80,"threads_started":1}
I20260619 22:23:49.793628 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:49.793202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:47155)
0619 22:23:49.793462 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:49.793531 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260619 22:23:50.443527 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:50.442772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:57077)
0619 22:23:50.443338 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:50.443414 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":117,"threads_started":1}
I20260619 22:23:50.594702 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:50.594000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:41385)
0619 22:23:50.594495 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:50.594578 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":88,"threads_started":1}
I20260619 22:23:50.795399 16663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:50.794783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:48235)
0619 22:23:50.795210 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:50.795290 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":168,"threads_started":1}
I20260619 22:23:51.444751 16664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:51.444338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:50525)
0619 22:23:51.444587 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:51.444639 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":87,"threads_started":1}
I20260619 22:23:51.596246 16665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:51.595794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:55603)
0619 22:23:51.596081 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:51.596144 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":76,"threads_started":1}
I20260619 22:23:51.796834 16666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:51.796405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:60871)
0619 22:23:51.796659 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:51.796711 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260619 22:23:51.948489 16667 raft_consensus.cc:482] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:51.948653 16667 raft_consensus.cc:504] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28a72f93f7034a0b9fb94b7ff7ebf250" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46213 } } peers { permanent_uuid: "86faf5b30df54955b5a6b356fa66767a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 43245 } } peers { permanent_uuid: "3e86444c2d9b4072ad499221928628ad" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45457 } }
I20260619 22:23:51.949086 16667 leader_election.cc:288] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213), 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245)
W20260619 22:23:51.950021 16242 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28a72f93f7034a0b9fb94b7ff7ebf250 (127.1.74.131:46213): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
W20260619 22:23:51.950141 16240 leader_election.cc:341] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86faf5b30df54955b5a6b356fa66767a (127.1.74.130:43245): Not found: Tablet not found: 45026e1cab54487094c5a56bfdcadf1f
I20260619 22:23:51.950276 16240 leader_election.cc:302] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e86444c2d9b4072ad499221928628ad; no voters: 28a72f93f7034a0b9fb94b7ff7ebf250, 86faf5b30df54955b5a6b356fa66767a
I20260619 22:23:51.950455 16667 raft_consensus.cc:2726] T 45026e1cab54487094c5a56bfdcadf1f P 3e86444c2d9b4072ad499221928628ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 22:23:52.446456 16668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:52.445736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:43981)
0619 22:23:52.446235 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:52.446308 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":87,"threads_started":1}
I20260619 22:23:52.597769 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:52.597208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:43573)
0619 22:23:52.597576 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:52.597649 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":105,"threads_started":1}
I20260619 22:23:52.798204 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:52.797743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:51631)
0619 22:23:52.798044 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:52.798104 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":120,"threads_started":1}
I20260619 22:23:53.447831 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:53.447369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:37235)
0619 22:23:53.447662 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:53.447725 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":78,"threads_started":1}
I20260619 22:23:53.599215 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:53.598705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:52351)
0619 22:23:53.599044 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:53.599107 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":95,"threads_started":1}
I20260619 22:23:53.799618 16673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:53.799187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.131:60713)
0619 22:23:53.799457 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:53.799519 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260619 22:23:54.449529 16674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:54.449066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.129:49899)
0619 22:23:54.449364 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:54.449428 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":82,"threads_started":1}
I20260619 22:23:54.600803 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:54.600306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:40885 (local address 127.1.74.130:53839)
0619 22:23:54.600627 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:54.600699 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:40885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":81,"threads_started":1}
I20260619 22:23:54.782534 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 16190
I20260619 22:23:54.790242 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 16357
I20260619 22:23:54.796662 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 16491
2026-06-19T22:23:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19270 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:23:54 dist-test-slave-k84h krb5kdc[16678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:23:54 dist-test-slave-k84h krb5kdc[16678](info): set up 2 sockets
Jun 19 22:23:54 dist-test-slave-k84h krb5kdc[16678](info): commencing operation
krb5kdc: starting...
W20260619 22:23:55.875167 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:23:55 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:23:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:23:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:23:56.007364 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44351
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:44351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:56.126153 16691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:56.126480 16691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:56.126528 16691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:56.131090 16691 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 22:23:56.131179 16691 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 22:23:56.131196 16691 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 22:23:56.131211 16691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:23:56.131230 16691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:56.131248 16691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:56.131265 16691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:23:56.131281 16691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:23:56.136792 16691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:44351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:56.137873 16691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:56.138955 16691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:56.147753 16697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:56.147724 16696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:56.147862 16691 server_base.cc:1034] running on GCE node
W20260619 22:23:56.147781 16699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:56.148551 16691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:56.149731 16691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:56.150941 16691 hybrid_clock.cc:648] HybridClock initialized: now 1781907836150913 us; error 40 us; skew 500 ppm
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:56.153793 16691 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:23:56.154678 16691 webserver.cc:460] Webserver started at http://127.1.74.190:40313/ using document root <none> and password file <none>
I20260619 22:23:56.155252 16691 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:56.155313 16691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:56.155483 16691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:56.157495 16691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "9877f97758634c95bbc4bc5eab2bcd2f"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "634701282f8c8f6ca29ee2533217e560"
server_key_iv: "8fe6b88adff3dfe42944bbaf9aa5a241"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.158005 16691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "9877f97758634c95bbc4bc5eab2bcd2f"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "634701282f8c8f6ca29ee2533217e560"
server_key_iv: "8fe6b88adff3dfe42944bbaf9aa5a241"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.161756 16691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 22:23:56.164270 16706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.165504 16691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:23:56.165652 16691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "9877f97758634c95bbc4bc5eab2bcd2f"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "634701282f8c8f6ca29ee2533217e560"
server_key_iv: "8fe6b88adff3dfe42944bbaf9aa5a241"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.165767 16691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:56.196147 16691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:56.199733 16691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:56.200081 16691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:56.208169 16691 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:44351
I20260619 22:23:56.208487 16758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:44351 every 8 connection(s)
I20260619 22:23:56.209183 16691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:23:56.212152 16759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:56.214565 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16691
I20260619 22:23:56.214721 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:23:56.215070 1322 external_mini_cluster.cc:1428] Setting key 496d2b0205a6a54688b4c879183dcf4a
I20260619 22:23:56.218616 16759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap starting.
I20260619 22:23:56.221416 16759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:56.222342 16759 log.cc:824] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:56.224546 16759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: No bootstrap required, opened a new log
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:56.229625 16759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:23:56.229933 16759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:56.230005 16759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9877f97758634c95bbc4bc5eab2bcd2f, State: Initialized, Role: FOLLOWER
I20260619 22:23:56.230508 16759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:23:56.230659 16759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:23:56.230726 16759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:23:56.230820 16759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:56.230890 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.216573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.1.74.190:44351)
0619 22:23:56.217018 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:56.217036 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:56.217079 (+ 43us) server_negotiation.cc:407] Connection header received
0619 22:23:56.217917 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:56.217941 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:56.218374 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:56.218739 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:56.219521 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.220993 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.221890 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.222591 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.222767 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.225334 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:56.225340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:56.225358 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:56.225391 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:56.227635 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.228116 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.228124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.228131 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.228283 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.228542 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.228546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.228548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.229123 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:56.229335 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:56.229527 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 22:23:56.229847 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":114,"threads_started":1}
I20260619 22:23:56.231851 16759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:23:56.232215 16759 leader_election.cc:302] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9877f97758634c95bbc4bc5eab2bcd2f; no voters:
I20260619 22:23:56.232545 16759 leader_election.cc:288] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:23:56.232661 16764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:56.232867 16764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 LEADER]: Becoming Leader. State: Replica: 9877f97758634c95bbc4bc5eab2bcd2f, State: Running, Role: LEADER
I20260619 22:23:56.233547 16764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:23:56.233883 16759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:23:56.235219 16765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } } }
I20260619 22:23:56.235352 16765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:56.235729 16773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:23:56.235677 16766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9877f97758634c95bbc4bc5eab2bcd2f. Latest consensus state: current_term: 1 leader_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } } }
I20260619 22:23:56.235868 16766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:23:56.238798 16773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:23:56.244799 16773 catalog_manager.cc:1347] Generated new cluster ID: 9eaa5a9eb65349d3a8572022a1fabecd
I20260619 22:23:56.244908 16773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:23:56.254828 16773 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:23:56.255949 16773 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:23:56.262439 16773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Generated new TSK 0
I20260619 22:23:56.263262 16773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:23:56.297809 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44351
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:56.416546 16785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:56.416872 16785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:56.416919 16785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:56.421607 16785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:56.421694 16785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:56.421808 16785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:23:56.427867 16785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:56.429085 16785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:56.430069 16785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:56.440642 16790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:56.440642 16793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:56.440637 16791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:56.441036 16785 server_base.cc:1034] running on GCE node
I20260619 22:23:56.441442 16785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:56.442135 16785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:56.443334 16785 hybrid_clock.cc:648] HybridClock initialized: now 1781907836443313 us; error 37 us; skew 500 ppm
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:56.446533 16785 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:23:56.447614 16785 webserver.cc:460] Webserver started at http://127.1.74.129:43107/ using document root <none> and password file <none>
I20260619 22:23:56.448235 16785 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:56.448302 16785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:56.448525 16785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:56.450728 16785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "3cc8e55042e89af0beffb398a7b31063"
server_key_iv: "9f2c55e26c52a33df4cb95e67e8ce258"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.451404 16785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "3cc8e55042e89af0beffb398a7b31063"
server_key_iv: "9f2c55e26c52a33df4cb95e67e8ce258"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.455536 16785 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:23:56.458079 16800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.459504 16785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:56.459650 16785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "3cc8e55042e89af0beffb398a7b31063"
server_key_iv: "9f2c55e26c52a33df4cb95e67e8ce258"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.459775 16785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:56.476781 16785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:56.480351 16785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:56.480577 16785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:56.481213 16785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:56.482088 16785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:56.482161 16785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.482235 16785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:56.482278 16785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.492195 16785 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:46705
I20260619 22:23:56.492252 16913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:46705 every 8 connection(s)
I20260619 22:23:56.493162 16785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:23:56.495363 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16785
I20260619 22:23:56.495527 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:23:56.495813 1322 external_mini_cluster.cc:1428] Setting key 16e2cf7a68c2b0da94d599b28d993a49
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:23:56.509465 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.495115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:56853 (local address 127.1.74.190:44351)
0619 22:23:56.495296 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:56.495301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:56.496583 (+ 1282us) server_negotiation.cc:407] Connection header received
0619 22:23:56.497561 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:56.497564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:56.497620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:56.497710 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:56.499161 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.499825 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.501821 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.502064 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.502172 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.505444 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:56.505450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:56.505457 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:56.505495 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:56.507210 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.507749 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.507751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.507752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.507826 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.508263 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.508265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.508266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.508469 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:56.508549 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:56.509166 (+ 617us) server_negotiation.cc:299] Negotiation successful
0619 22:23:56.509281 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:23:56.510547 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.495642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:56853)
0619 22:23:56.496422 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:56.496478 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:56.497340 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:56.497889 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:56.497897 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:56.498285 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:56.498905 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:56.498934 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.499994 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.500001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:56.501602 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:56.501620 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.502386 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.502390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:56.502571 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.503317 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:56.503334 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:56.505191 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:56.507342 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:56.507349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:56.507365 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:56.507638 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:56.507985 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:56.507991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:56.507996 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:56.508172 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:56.508571 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:56.508577 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:56.508917 (+ 340us) client_negotiation.cc:769] Sending connection context
0619 22:23:56.509109 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 22:23:56.509409 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":146,"threads_started":1}
I20260619 22:23:56.512166 16914 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44351
I20260619 22:23:56.512450 16914 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:56.512992 16914 heartbeater.cc:511] Master 127.1.74.190:44351 requested a full tablet report, sending...
I20260619 22:23:56.515182 16721 ts_manager.cc:194] Registered new tserver with Master: 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705)
I20260619 22:23:56.516359 16721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:56853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:23:56.527001 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44351
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:56.648350 16919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:56.648656 16919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:56.648702 16919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:56.652858 16919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:56.652935 16919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:56.652998 16919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:23:56.658495 16919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:56.659710 16919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:56.660737 16919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:56.671628 16927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:56.671628 16925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:56.671658 16924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:56.672049 16919 server_base.cc:1034] running on GCE node
I20260619 22:23:56.672482 16919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:56.673166 16919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:56.674417 16919 hybrid_clock.cc:648] HybridClock initialized: now 1781907836674366 us; error 73 us; skew 500 ppm
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:56.677764 16919 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:23:56.678948 16919 webserver.cc:460] Webserver started at http://127.1.74.130:40115/ using document root <none> and password file <none>
I20260619 22:23:56.679574 16919 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:56.679668 16919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:56.679915 16919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:56.682178 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "be96f307cdc6428cb457c89ce4a29b55"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "57f57e96dd267ccbc89319cfafb4f7ae"
server_key_iv: "d96c13be278d5db0fc692749dfde18bd"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.682802 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "be96f307cdc6428cb457c89ce4a29b55"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "57f57e96dd267ccbc89319cfafb4f7ae"
server_key_iv: "d96c13be278d5db0fc692749dfde18bd"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.687080 16919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:23:56.689843 16934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.691489 16919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:23:56.691653 16919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "be96f307cdc6428cb457c89ce4a29b55"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "57f57e96dd267ccbc89319cfafb4f7ae"
server_key_iv: "d96c13be278d5db0fc692749dfde18bd"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.691804 16919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:56.717365 16919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:56.720883 16919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:56.721118 16919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:56.721745 16919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:56.722618 16919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:56.722692 16919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.722779 16919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:56.722822 16919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.733618 16919 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:45193
I20260619 22:23:56.733637 17047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:45193 every 8 connection(s)
I20260619 22:23:56.734684 16919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:23:56.744719 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 16919
I20260619 22:23:56.744850 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:56.745251 1322 external_mini_cluster.cc:1428] Setting key 7ddf54bcf70c56e1e2b933e5859edd84
I20260619 22:23:56.750247 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.736870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:51009 (local address 127.1.74.190:44351)
0619 22:23:56.737025 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:56.737030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:56.738120 (+ 1090us) server_negotiation.cc:407] Connection header received
0619 22:23:56.739085 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:56.739088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:56.739139 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:56.739221 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:56.740688 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.741265 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.742517 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.742774 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.742953 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.745877 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:56.745881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:56.745883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:56.745914 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:56.747704 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.748469 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.748471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.748473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.748557 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.748986 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.748991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.748993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.749216 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:56.749328 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:56.749982 (+ 654us) server_negotiation.cc:299] Negotiation successful
0619 22:23:56.750107 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 22:23:56.751571 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.737274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:51009)
0619 22:23:56.737973 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:56.738016 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:56.738807 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:56.739504 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:56.739516 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:56.739908 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:56.740493 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:56.740508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.741430 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.741433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:56.742355 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:56.742365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.743000 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.743003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:56.743158 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.743817 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:56.743834 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:56.745661 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:56.747855 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:56.747866 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:56.747890 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:56.748339 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:56.748693 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:56.748696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:56.748698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:56.748880 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:56.749326 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:56.749334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:56.749618 (+ 284us) client_negotiation.cc:769] Sending connection context
0619 22:23:56.749890 (+ 272us) client_negotiation.cc:240] Negotiation successful
0619 22:23:56.750299 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":129,"threads_started":1}
I20260619 22:23:56.753119 17048 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44351
I20260619 22:23:56.753415 17048 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:56.753996 17048 heartbeater.cc:511] Master 127.1.74.190:44351 requested a full tablet report, sending...
I20260619 22:23:56.755752 16721 ts_manager.cc:194] Registered new tserver with Master: be96f307cdc6428cb457c89ce4a29b55 (127.1.74.130:45193)
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:23:56.756949 16721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:51009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:23:56.778384 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44351
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:23:56.896977 17053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:23:56.897298 17053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:23:56.897346 17053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:23:56.901571 17053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:23:56.901676 17053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:23:56.901743 17053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:23:56.907287 17053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:23:56.908380 17053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:23:56.909368 17053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:23:56.920612 17061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:23:56.920660 17058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:56.920850 17053 server_base.cc:1034] running on GCE node
W20260619 22:23:56.920612 17059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:23:56.921401 17053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:23:56.922148 17053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:23:56.923352 17053 hybrid_clock.cc:648] HybridClock initialized: now 1781907836923321 us; error 44 us; skew 500 ppm
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:23:56.926378 17053 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:23:56.927371 17053 webserver.cc:460] Webserver started at http://127.1.74.131:37585/ using document root <none> and password file <none>
I20260619 22:23:56.927918 17053 fs_manager.cc:359] Metadata directory not provided
I20260619 22:23:56.927980 17053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:23:56.928148 17053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:23:56.930655 17053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "f11462123b6c400ba3f1b62c7ce87ba5"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "38a2052aa204bb774cd62f8359341c35"
server_key_iv: "7f3c30078ee3c95fe53bab79bdb70646"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.931429 17053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "f11462123b6c400ba3f1b62c7ce87ba5"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "38a2052aa204bb774cd62f8359341c35"
server_key_iv: "7f3c30078ee3c95fe53bab79bdb70646"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.936466 17053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 22:23:56.940653 17068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.942669 17053 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:23:56.942926 17053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "f11462123b6c400ba3f1b62c7ce87ba5"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "38a2052aa204bb774cd62f8359341c35"
server_key_iv: "7f3c30078ee3c95fe53bab79bdb70646"
server_key_version: "encryptionkey@0"
I20260619 22:23:56.943112 17053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:23:56.957141 17053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:23:56.960959 17053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:23:56.961153 17053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:23:56.961761 17053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:23:56.962930 17053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:23:56.963014 17053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.963096 17053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:23:56.963132 17053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:23:56.974027 17053 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:38557
I20260619 22:23:56.974048 17181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:38557 every 8 connection(s)
I20260619 22:23:56.975364 17053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:23:56.976147 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17053
I20260619 22:23:56.976334 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:23:56.976886 1322 external_mini_cluster.cc:1428] Setting key 12882f00882e915d66fc05a9731e361f
Jun 19 22:23:56 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907836, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:23:56.995743 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.978199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:54679 (local address 127.1.74.190:44351)
0619 22:23:56.978380 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:56.978385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:56.980207 (+ 1822us) server_negotiation.cc:407] Connection header received
0619 22:23:56.981969 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:56.981971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:56.982032 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:56.982113 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:56.984102 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.984702 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.986181 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.986504 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.986674 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.990461 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:56.990472 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:56.990475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:56.990518 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:56.992422 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.993063 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.993066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.993067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.993150 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:56.993532 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:56.993535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:56.993536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:56.993793 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:56.993883 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:56.995345 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0619 22:23:56.995488 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 22:23:56.996913 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:56.979114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:54679)
0619 22:23:56.979956 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:56.980050 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:56.981667 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:56.982464 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:56.982476 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:56.983245 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:23:56.983900 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:56.983925 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.984880 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.984883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:56.985935 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:56.985946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:56.986760 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:56.986766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:56.987047 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:56.987792 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:23:56.987811 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:23:56.990166 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:23:56.992572 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:56.992579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:56.992598 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:56.992946 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:56.993236 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:23:56.993239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:23:56.993240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:23:56.993400 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:23:56.994021 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:23:56.994032 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:23:56.994692 (+ 660us) client_negotiation.cc:769] Sending connection context
0619 22:23:56.995273 (+ 581us) client_negotiation.cc:240] Negotiation successful
0619 22:23:56.995824 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":264,"threads_started":1}
I20260619 22:23:56.998730 17182 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44351
I20260619 22:23:56.999070 17182 heartbeater.cc:464] Registering TS with master...
I20260619 22:23:56.999668 17182 heartbeater.cc:511] Master 127.1.74.190:44351 requested a full tablet report, sending...
I20260619 22:23:57.000909 16721 ts_manager.cc:194] Registered new tserver with Master: f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557)
I20260619 22:23:57.001543 16721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:54679
I20260619 22:23:57.004447 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:23:57.016309 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.007603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.1.74.190:44351)
0619 22:23:57.007849 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.007854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.008045 (+ 191us) server_negotiation.cc:407] Connection header received
0619 22:23:57.008231 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.008235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.008291 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.008395 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:23:57.009205 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.009777 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.010689 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.011021 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.011150 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.012821 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:23:57.012825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:23:57.012828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:23:57.012867 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:23:57.014393 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:57.014924 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:57.014928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:57.014930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:57.015022 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:23:57.015327 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:23:57.015330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:23:57.015332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:23:57.015584 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:23:57.015736 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.015991 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.016118 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 22:23:57.019685 16721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:23:57.023478 16721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:23:57.041999 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.034696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.1.74.129:46705)
0619 22:23:57.035005 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.035014 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.035175 (+ 161us) server_negotiation.cc:407] Connection header received
0619 22:23:57.035478 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.035485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.035719 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.035893 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:57.036335 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.037720 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.040307 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.041044 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.041193 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.041363 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.041681 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.041822 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
I20260619 22:23:57.041991 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.033864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.0.0.1:57144)
0619 22:23:57.034955 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.035062 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:57.035230 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:57.036056 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:57.036058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:57.036095 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:57.036206 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.036213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.038953 (+ 2740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.038956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.040122 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.040135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.041194 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.041198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.041448 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.041483 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 22:23:57.041627 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 22:23:57.041840 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":137,"thread_start_us":91,"threads_started":1}
I20260619 22:23:57.042354 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.034027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:38557 (local address 127.0.0.1:58332)
0619 22:23:57.035175 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.035191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:57.035292 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:57.035898 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:57.035905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:57.035928 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:57.036081 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.036088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.037485 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.037489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.039558 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.039585 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.041530 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.041533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.041632 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.041651 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 22:23:57.041713 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 22:23:57.042234 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":569,"thread_start_us":110,"threads_started":1}
I20260619 22:23:57.042002 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.034163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45193 (local address 127.0.0.1:49654)
0619 22:23:57.035508 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.035522 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:57.035601 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:57.036380 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:57.036384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:57.036402 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:57.036493 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.036498 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.037853 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.037857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.039558 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.039585 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.041161 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.041165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.041432 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.041484 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 22:23:57.041598 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 22:23:57.041818 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291,"mutex_wait_us":102,"thread_start_us":110,"threads_started":1}
I20260619 22:23:57.042039 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.034675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.1.74.131:38557)
0619 22:23:57.035428 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.035440 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.035464 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:23:57.035529 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.035534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.035718 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.036260 (+ 542us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:57.036424 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.037307 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.040337 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.041178 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.041311 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.041465 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.041797 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.041897 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":175,"threads_started":1}
I20260619 22:23:57.043188 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.034326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.1.74.130:45193)
0619 22:23:57.035859 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.035872 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.035894 (+ 22us) server_negotiation.cc:407] Connection header received
0619 22:23:57.035975 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.035980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.036226 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.036419 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:57.036580 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.037720 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.039810 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.040782 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.040901 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.041080 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.042883 (+ 1803us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.043016 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1419,"thread_start_us":116,"threads_started":1}
I20260619 22:23:57.045830 17116 tablet_service.cc:1467] Processing CreateTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e (DEFAULT_TABLE table=test-table [id=3696a185470c41c1bf9036cacc767d20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:57.045830 16848 tablet_service.cc:1467] Processing CreateTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e (DEFAULT_TABLE table=test-table [id=3696a185470c41c1bf9036cacc767d20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:57.047188 16848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd2b800ce2d41fa8c32783eb7f0d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:57.047194 17116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd2b800ce2d41fa8c32783eb7f0d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:57.050156 16982 tablet_service.cc:1467] Processing CreateTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e (DEFAULT_TABLE table=test-table [id=3696a185470c41c1bf9036cacc767d20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:23:57.051352 16982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd2b800ce2d41fa8c32783eb7f0d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:23:57.058698 17199 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap starting.
I20260619 22:23:57.059962 17200 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Bootstrap starting.
I20260619 22:23:57.060070 17201 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Bootstrap starting.
I20260619 22:23:57.062502 17199 tablet_bootstrap.cc:654] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:57.062960 17201 tablet_bootstrap.cc:654] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:57.063975 17199 log.cc:824] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:57.063994 17201 log.cc:824] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:57.065773 17200 tablet_bootstrap.cc:654] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Neither blocks nor log segments found. Creating new log.
I20260619 22:23:57.066357 17201 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: No bootstrap required, opened a new log
I20260619 22:23:57.066597 17201 ts_tablet_manager.cc:1397] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260619 22:23:57.066633 17200 log.cc:824] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Log is configured to *not* fsync() on all Append() calls
I20260619 22:23:57.068410 17199 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: No bootstrap required, opened a new log
I20260619 22:23:57.068742 17199 ts_tablet_manager.cc:1397] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.002s
I20260619 22:23:57.068774 17200 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: No bootstrap required, opened a new log
I20260619 22:23:57.068941 17200 ts_tablet_manager.cc:1397] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260619 22:23:57.071462 17201 raft_consensus.cc:348] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.071880 17201 raft_consensus.cc:374] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:57.071944 17201 raft_consensus.cc:729] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be96f307cdc6428cb457c89ce4a29b55, State: Initialized, Role: FOLLOWER
I20260619 22:23:57.072571 17201 consensus_queue.cc:260] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.073531 17048 heartbeater.cc:503] Master 127.1.74.190:44351 was elected leader, sending a full tablet report...
I20260619 22:23:57.073588 17199 raft_consensus.cc:348] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.073463 17200 raft_consensus.cc:348] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.073855 17199 raft_consensus.cc:374] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:57.073855 17200 raft_consensus.cc:374] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:23:57.073909 17199 raft_consensus.cc:729] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fc1aa19ebd48ca97cdf37069f43f66, State: Initialized, Role: FOLLOWER
I20260619 22:23:57.073909 17200 raft_consensus.cc:729] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11462123b6c400ba3f1b62c7ce87ba5, State: Initialized, Role: FOLLOWER
I20260619 22:23:57.074047 17201 ts_tablet_manager.cc:1428] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260619 22:23:57.074553 17199 consensus_queue.cc:260] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.074553 17200 consensus_queue.cc:260] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.075893 17200 ts_tablet_manager.cc:1428] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Time spent starting tablet: real 0.007s user 0.000s sys 0.008s
I20260619 22:23:57.075893 17199 ts_tablet_manager.cc:1428] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260619 22:23:57.076372 16914 heartbeater.cc:503] Master 127.1.74.190:44351 was elected leader, sending a full tablet report...
I20260619 22:23:57.076706 17182 heartbeater.cc:503] Master 127.1.74.190:44351 was elected leader, sending a full tablet report...
W20260619 22:23:57.227167 17183 tablet.cc:2369] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:23:57.236088 17049 tablet.cc:2369] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:23:57.245045 16915 tablet.cc:2369] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:23:57.337561 17205 raft_consensus.cc:482] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:23:57.337824 17205 raft_consensus.cc:504] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.339169 17205 leader_election.cc:288] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705), f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557)
I20260619 22:23:57.343524 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.339633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:50029 (local address 127.1.74.131:38557)
0619 22:23:57.339812 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.339817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.340054 (+ 237us) server_negotiation.cc:407] Connection header received
0619 22:23:57.340180 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.340184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.340241 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.340326 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:57.340671 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.341146 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.342437 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.342922 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.343018 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.343091 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.343306 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.343385 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 22:23:57.343549 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.339381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.1.74.130:37823)
0619 22:23:57.339791 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.339810 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:57.339929 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:57.340193 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:57.340196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:57.340214 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:57.340420 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.340429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.341245 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.341249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.342316 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.342326 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.343128 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.343132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.343212 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.343229 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 22:23:57.343329 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 22:23:57.343396 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":99,"thread_start_us":96,"threads_started":1}
I20260619 22:23:57.343554 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.339790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:37823 (local address 127.1.74.129:46705)
0619 22:23:57.339938 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.339943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.339962 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:23:57.340024 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.340027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.340081 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.340160 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:23:57.340552 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.341112 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.342471 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.342993 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.343118 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.343172 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.343363 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.343446 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:23:57.343559 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.339428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:38557 (local address 127.1.74.130:50029)
0619 22:23:57.339920 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.339935 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:23:57.340051 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:23:57.340345 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:23:57.340347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:23:57.340362 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:23:57.340540 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.340547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.341295 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.341297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.342312 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:23:57.342321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.343040 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.343044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:23:57.343146 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.343172 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 22:23:57.343260 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 22:23:57.343358 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":54,"thread_start_us":85,"threads_started":1}
I20260619 22:23:57.344285 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd2b800ce2d41fa8c32783eb7f0d72e" candidate_uuid: "be96f307cdc6428cb457c89ce4a29b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" is_pre_election: true
I20260619 22:23:57.344650 16868 raft_consensus.cc:2454] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be96f307cdc6428cb457c89ce4a29b55 in term 0.
I20260619 22:23:57.344650 17136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd2b800ce2d41fa8c32783eb7f0d72e" candidate_uuid: "be96f307cdc6428cb457c89ce4a29b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" is_pre_election: true
I20260619 22:23:57.344924 17136 raft_consensus.cc:2454] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be96f307cdc6428cb457c89ce4a29b55 in term 0.
I20260619 22:23:57.345198 16935 leader_election.cc:302] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fc1aa19ebd48ca97cdf37069f43f66, be96f307cdc6428cb457c89ce4a29b55; no voters:
I20260619 22:23:57.345535 17205 raft_consensus.cc:2781] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:23:57.345628 17205 raft_consensus.cc:482] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:23:57.345672 17205 raft_consensus.cc:3037] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:57.346817 17205 raft_consensus.cc:504] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.347312 17205 leader_election.cc:288] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 1 election: Requested vote from peers 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705), f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557)
I20260619 22:23:57.347716 16868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd2b800ce2d41fa8c32783eb7f0d72e" candidate_uuid: "be96f307cdc6428cb457c89ce4a29b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
I20260619 22:23:57.347743 17136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd2b800ce2d41fa8c32783eb7f0d72e" candidate_uuid: "be96f307cdc6428cb457c89ce4a29b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f11462123b6c400ba3f1b62c7ce87ba5"
I20260619 22:23:57.347857 17136 raft_consensus.cc:3037] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:57.347858 16868 raft_consensus.cc:3037] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:23:57.349187 16868 raft_consensus.cc:2454] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be96f307cdc6428cb457c89ce4a29b55 in term 1.
I20260619 22:23:57.349187 17136 raft_consensus.cc:2454] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be96f307cdc6428cb457c89ce4a29b55 in term 1.
I20260619 22:23:57.349499 16935 leader_election.cc:302] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fc1aa19ebd48ca97cdf37069f43f66, be96f307cdc6428cb457c89ce4a29b55; no voters:
I20260619 22:23:57.349653 17205 raft_consensus.cc:2781] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:23:57.349884 17205 raft_consensus.cc:686] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 1 LEADER]: Becoming Leader. State: Replica: be96f307cdc6428cb457c89ce4a29b55, State: Running, Role: LEADER
I20260619 22:23:57.350212 17205 consensus_queue.cc:237] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:23:57.353629 16719 catalog_manager.cc:5495] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 reported cstate change: term changed from 0 to 1, leader changed from <none> to be96f307cdc6428cb457c89ce4a29b55 (127.1.74.130). New cstate: current_term: 1 leader_uuid: "be96f307cdc6428cb457c89ce4a29b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } health_report { overall_health: HEALTHY } } }
I20260619 22:23:57.420428 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.416732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.1.74.130:45193)
0619 22:23:57.416891 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 22:23:57.416898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:23:57.416915 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:23:57.417013 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:23:57.417015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:23:57.417071 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:23:57.417197 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:23:57.417449 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.417967 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.418897 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:23:57.419108 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:23:57.419225 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:23:57.419545 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:23:57.420002 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:23:57.420145 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 22:23:57.420201 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 22:23:57.420275 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:23:57.427366 17136 raft_consensus.cc:1261] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Refusing update from remote peer be96f307cdc6428cb457c89ce4a29b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:57.427366 16868 raft_consensus.cc:1261] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Refusing update from remote peer be96f307cdc6428cb457c89ce4a29b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:23:57.428222 17210 consensus_queue.cc:1035] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:57.428437 17205 consensus_queue.cc:1035] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:23:57.438333 17215 mvcc.cc:205] Tried to move back new op lower bound from 7298694502093234176 to 7298694501789257728. Current Snapshot: MvccSnapshot[applied={T|T < 7298694502093234176}]
I20260619 22:23:57.440349 17213 mvcc.cc:205] Tried to move back new op lower bound from 7298694502093234176 to 7298694501789257728. Current Snapshot: MvccSnapshot[applied={T|T < 7298694502093234176 or (T in {7298694502093234176})}]
I20260619 22:23:57.440493 17214 mvcc.cc:205] Tried to move back new op lower bound from 7298694502093234176 to 7298694501789257728. Current Snapshot: MvccSnapshot[applied={T|T < 7298694502093234176}]
I20260619 22:23:57.450363 16721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:57.450587 16721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:23:57.453362 16721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Sending DeleteTablet for 3 replicas of tablet fcd2b800ce2d41fa8c32783eb7f0d72e
I20260619 22:23:57.454543 16848 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:57144
I20260619 22:23:57.454624 17116 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:58332
I20260619 22:23:57.454653 16982 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:49654
I20260619 22:23:57.455188 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 16785
I20260619 22:23:57.456087 17222 tablet_replica.cc:330] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: stopping tablet replica
I20260619 22:23:57.456497 17222 raft_consensus.cc:2229] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:23:57.456876 17222 raft_consensus.cc:2258] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:57.457336 17224 tablet_replica.cc:330] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: stopping tablet replica
I20260619 22:23:57.457670 17224 raft_consensus.cc:2229] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:23:57.458114 17224 raft_consensus.cc:2258] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:23:57.459035 17222 ts_tablet_manager.cc:1905] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:57.459630 17224 ts_tablet_manager.cc:1905] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:23:57.462234 17222 ts_tablet_manager.cc:1918] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:57.462445 17222 log.cc:1196] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/fcd2b800ce2d41fa8c32783eb7f0d72e
I20260619 22:23:57.462879 17222 ts_tablet_manager.cc:1939] T fcd2b800ce2d41fa8c32783eb7f0d72e P f11462123b6c400ba3f1b62c7ce87ba5: Deleting consensus metadata
I20260619 22:23:57.463382 17224 ts_tablet_manager.cc:1918] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:23:57.463485 17224 log.cc:1196] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/fcd2b800ce2d41fa8c32783eb7f0d72e
I20260619 22:23:57.463805 17224 ts_tablet_manager.cc:1939] T fcd2b800ce2d41fa8c32783eb7f0d72e P be96f307cdc6428cb457c89ce4a29b55: Deleting consensus metadata
W20260619 22:23:57.464114 16707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:46705 (error 108)
I20260619 22:23:57.464614 16709 catalog_manager.cc:4841] TS f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557): tablet fcd2b800ce2d41fa8c32783eb7f0d72e (table test-table [id=3696a185470c41c1bf9036cacc767d20]) successfully deleted
I20260619 22:23:57.465076 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 16919
I20260619 22:23:57.470252 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.469828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.0.0.1:57154)
0619 22:23:57.470031 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.470138 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260619 22:23:57.470494 16707 catalog_manager.cc:4584] TS 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fcd2b800ce2d41fa8c32783eb7f0d72e: Network error: Client connection negotiation failed: client connection to 127.1.74.129:46705: connect: Connection refused (error 111)
I20260619 22:23:57.471380 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17053
I20260619 22:23:57.473647 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:23:57.471555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45193 (local address 127.0.0.1:49674)
0619 22:23:57.473450 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0619 22:23:57.473526 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:45193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1778}
I20260619 22:23:57.477306 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 16691
I20260619 22:24:17.484581 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44351
--webserver_interface=127.1.74.190
--webserver_port=40313
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:44351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:17.604676 17226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:17.605064 17226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:17.605137 17226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:17.609789 17226 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 22:24:17.609882 17226 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 22:24:17.609901 17226 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 22:24:17.609916 17226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:24:17.609935 17226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:17.609954 17226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:17.609972 17226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:24:17.609987 17226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:24:17.616190 17226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:44351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=40313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:17.617468 17226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:17.618613 17226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:17.632541 17233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:17.632541 17235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:17.632586 17232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:17.633106 17226 server_base.cc:1034] running on GCE node
I20260619 22:24:17.633780 17226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:17.635103 17226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:17.636511 17226 hybrid_clock.cc:648] HybridClock initialized: now 1781907857636462 us; error 79 us; skew 500 ppm
Jun 19 22:24:17 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:17.639690 17226 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:24:17.640781 17226 webserver.cc:460] Webserver started at http://127.1.74.190:40313/ using document root <none> and password file <none>
I20260619 22:24:17.641399 17226 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:17.641511 17226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:17.645902 17226 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:24:17.648484 17242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:17.649967 17226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:24:17.650138 17226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "9877f97758634c95bbc4bc5eab2bcd2f"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "634701282f8c8f6ca29ee2533217e560"
server_key_iv: "8fe6b88adff3dfe42944bbaf9aa5a241"
server_key_version: "encryptionkey@0"
I20260619 22:24:17.650811 17226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:17.682618 17226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:17.686131 17226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:17.686363 17226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:17.694715 17226 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:44351
I20260619 22:24:17.695065 17294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:44351 every 8 connection(s)
I20260619 22:24:17.696099 17226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:24:17.701139 17295 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:24:17.703248 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17226
I20260619 22:24:17.703281 17295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap starting.
I20260619 22:24:17.703771 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:46705
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:44351
--webserver_port=43107
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:24:17.722512 17295 log.cc:824] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:17.732985 17295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:24:17.733461 17295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap complete.
I20260619 22:24:17.737888 17295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:17.738327 17295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9877f97758634c95bbc4bc5eab2bcd2f, State: Initialized, Role: FOLLOWER
I20260619 22:24:17.738942 17295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:17.739070 17295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:24:17.739164 17295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:24:17.739282 17295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:24:17.741662 17295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:17.742046 17295 leader_election.cc:302] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9877f97758634c95bbc4bc5eab2bcd2f; no voters:
I20260619 22:24:17.742401 17295 leader_election.cc:288] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 22:24:17.742504 17299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Leader election won for term 2
I20260619 22:24:17.742770 17299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 LEADER]: Becoming Leader. State: Replica: 9877f97758634c95bbc4bc5eab2bcd2f, State: Running, Role: LEADER
I20260619 22:24:17.743144 17299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:17.743835 17295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:24:17.744750 17301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9877f97758634c95bbc4bc5eab2bcd2f. Latest consensus state: current_term: 2 leader_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } } }
I20260619 22:24:17.744779 17300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } } }
I20260619 22:24:17.744901 17300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:17.744900 17301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:17.749965 17314 catalog_manager.cc:1259] Loaded cluster ID: 9eaa5a9eb65349d3a8572022a1fabecd
I20260619 22:24:17.750075 17314 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: loading cluster ID for follower catalog manager: success
I20260619 22:24:17.752888 17314 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: acquiring CA information for follower catalog manager: success
I20260619 22:24:17.755559 17314 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:24:17.755762 17308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:24:17.758533 17308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:24:17.759074 17308 catalog_manager.cc:1259] Loaded cluster ID: 9eaa5a9eb65349d3a8572022a1fabecd
I20260619 22:24:17.759230 17308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:24:17.759979 17308 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:24:17.760475 17308 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Loaded TSK: 0
I20260619 22:24:17.778168 17308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Generated new TSK 1
I20260619 22:24:17.778507 17308 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 22:24:17.839282 17297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:17.839607 17297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:17.839681 17297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:17.845057 17297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:17.845155 17297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:17.845257 17297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:24:17.851495 17297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:46705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=43107
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:17.852814 17297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:17.853984 17297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:17.866159 17322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:17.866214 17323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:17.866181 17325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:17.866482 17297 server_base.cc:1034] running on GCE node
I20260619 22:24:17.867235 17297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:17.868141 17297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:17.869457 17297 hybrid_clock.cc:648] HybridClock initialized: now 1781907857869409 us; error 59 us; skew 500 ppm
Jun 19 22:24:17 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:17.873389 17297 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:24:17.874524 17297 webserver.cc:460] Webserver started at http://127.1.74.129:43107/ using document root <none> and password file <none>
I20260619 22:24:17.875211 17297 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:17.875326 17297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:17.879701 17297 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260619 22:24:17.882170 17332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:17.883713 17297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:24:17.883867 17297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "3cc8e55042e89af0beffb398a7b31063"
server_key_iv: "9f2c55e26c52a33df4cb95e67e8ce258"
server_key_version: "encryptionkey@0"
I20260619 22:24:17.884510 17297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:17.912259 17297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:17.915899 17297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:17.916126 17297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:17.916779 17297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:17.917969 17339 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 22:24:17.922341 17297 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 22:24:17.922461 17297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260619 22:24:17.922519 17297 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 22:24:17.924376 17297 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 22:24:17.924436 17297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260619 22:24:17.924764 17339 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap starting.
I20260619 22:24:17.935812 17297 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:46705
I20260619 22:24:17.935889 17446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:46705 every 8 connection(s)
I20260619 22:24:17.936918 17297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:24:17.943039 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17297
I20260619 22:24:17.943593 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:45193
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:44351
--webserver_port=40115
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:24:17.945992 17339 log.cc:824] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:24:17 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:17.955955 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:17.939363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:52821)
0619 22:24:17.940100 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:17.940155 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:17.941031 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:17.942635 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:17.942706 (+ 71us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:17.943295 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:17.943980 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:17.944006 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.946037 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.946042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:17.947359 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:17.947370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.948191 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.948196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:17.948379 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:17.949014 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:17.949031 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:17.950503 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:17.952698 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:17.952704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:17.952716 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:17.952978 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:17.953344 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:17.953348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:17.953350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:17.953484 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:17.954109 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:17.954115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:17.954369 (+ 254us) client_negotiation.cc:769] Sending connection context
0619 22:24:17.954557 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 22:24:17.954826 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":160,"threads_started":1}
I20260619 22:24:17.956471 17339 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:24:17.956710 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:17.939520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:52821 (local address 127.1.74.190:44351)
0619 22:24:17.939928 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:17.939947 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:17.940316 (+ 369us) server_negotiation.cc:407] Connection header received
0619 22:24:17.941550 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:17.941572 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:17.942144 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:17.942553 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:17.944214 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.945847 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.947498 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.948027 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.948230 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:17.950719 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:17.950724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:17.950737 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:17.950766 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:17.952532 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:17.953114 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:17.953128 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:17.953133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:17.953249 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:17.953584 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:17.953588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:17.953589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:17.953997 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:17.954136 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:17.955050 (+ 914us) server_negotiation.cc:299] Negotiation successful
0619 22:24:17.955337 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":133,"threads_started":1}
I20260619 22:24:17.956938 17339 tablet_bootstrap.cc:492] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap complete.
I20260619 22:24:17.958400 17339 ts_tablet_manager.cc:1397] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260619 22:24:17.958762 17447 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44351
I20260619 22:24:17.959002 17447 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:17.959527 17447 heartbeater.cc:511] Master 127.1.74.190:44351 requested a full tablet report, sending...
I20260619 22:24:17.961794 17259 ts_manager.cc:194] Registered new tserver with Master: 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705)
I20260619 22:24:17.961445 17339 raft_consensus.cc:348] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:24:17.961972 17339 raft_consensus.cc:729] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fc1aa19ebd48ca97cdf37069f43f66, State: Initialized, Role: FOLLOWER
I20260619 22:24:17.962466 17339 consensus_queue.cc:260] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } }
I20260619 22:24:17.963660 17339 ts_tablet_manager.cc:1428] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260619 22:24:17.965508 17259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:52821
I20260619 22:24:17.967849 17447 heartbeater.cc:503] Master 127.1.74.190:44351 was elected leader, sending a full tablet report...
Jun 19 22:24:17 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907857, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
I20260619 22:24:17.976830 17454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:17.964820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.0.0.1:48914)
0619 22:24:17.965537 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:17.965610 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:17.965757 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:17.966277 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:17.966283 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:17.966739 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:17.966897 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:17.966905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.968366 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.968370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:17.969355 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:17.969367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.970154 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.970157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:17.970244 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:17.970834 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:17.970888 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:17.972633 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:17.975038 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:17.975047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:17.975052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:17.975316 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:17.975650 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:17.975652 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:17.975653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:17.975740 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:17.976264 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:17.976269 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:17.976377 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 22:24:17.976501 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 22:24:17.976673 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":115,"threads_started":1}
I20260619 22:24:17.976964 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:17.964913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.1.74.129:46705)
0619 22:24:17.965281 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:17.965293 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:17.965711 (+ 418us) server_negotiation.cc:407] Connection header received
0619 22:24:17.965908 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:17.965914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:17.966112 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:17.966368 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:17.967038 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.968211 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.969507 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:17.970013 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:17.970123 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:17.972813 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:17.972826 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:17.972839 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:17.972865 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:17.974917 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:17.975438 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:17.975445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:17.975446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:17.975545 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:17.975824 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:17.975827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:17.975829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:17.976153 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:17.976283 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:17.976607 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 22:24:17.976791 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":96,"threads_started":1}
I20260619 22:24:17.977999 17381 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd2b800ce2d41fa8c32783eb7f0d72e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:23:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:48914
I20260619 22:24:17.978981 17456 tablet_replica.cc:330] stopping tablet replica
I20260619 22:24:17.979249 17456 raft_consensus.cc:2229] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:24:17.979467 17456 raft_consensus.cc:2258] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:17.980428 17456 ts_tablet_manager.cc:1905] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:24:17.984081 17456 ts_tablet_manager.cc:1918] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:24:17.984175 17456 log.cc:1196] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal/wals/fcd2b800ce2d41fa8c32783eb7f0d72e
I20260619 22:24:17.984382 17456 ts_tablet_manager.cc:1939] T fcd2b800ce2d41fa8c32783eb7f0d72e P 56fc1aa19ebd48ca97cdf37069f43f66: Deleting consensus metadata
I20260619 22:24:17.986464 17243 catalog_manager.cc:4841] TS 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705): tablet fcd2b800ce2d41fa8c32783eb7f0d72e (table test-table [id=3696a185470c41c1bf9036cacc767d20]) successfully deleted
W20260619 22:24:18.082230 17451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:18.082624 17451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:18.082701 17451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:18.087102 17451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:18.087219 17451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:18.087320 17451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:24:18.093338 17451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:45193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=40115
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:18.094583 17451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:18.095708 17451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:18.107100 17464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:18.107110 17462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:18.107156 17461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:18.107396 17451 server_base.cc:1034] running on GCE node
I20260619 22:24:18.108083 17451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:18.108856 17451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:18.110057 17451 hybrid_clock.cc:648] HybridClock initialized: now 1781907858110037 us; error 38 us; skew 500 ppm
Jun 19 22:24:18 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:18.113762 17451 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:24:18.114931 17451 webserver.cc:460] Webserver started at http://127.1.74.130:40115/ using document root <none> and password file <none>
I20260619 22:24:18.115571 17451 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:18.115689 17451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:18.120147 17451 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:24:18.123123 17471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:18.124806 17451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:24:18.124965 17451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "be96f307cdc6428cb457c89ce4a29b55"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "57f57e96dd267ccbc89319cfafb4f7ae"
server_key_iv: "d96c13be278d5db0fc692749dfde18bd"
server_key_version: "encryptionkey@0"
I20260619 22:24:18.125643 17451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:18.142932 17451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:18.147145 17451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:18.147403 17451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:18.148089 17451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:18.149073 17451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:18.149154 17451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:18.149233 17451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:18.149276 17451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:18.160476 17451 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:45193
I20260619 22:24:18.160519 17584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:45193 every 8 connection(s)
I20260619 22:24:18.161612 17451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:24:18.165767 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17451
I20260619 22:24:18.166402 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:38557
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:44351
--webserver_port=37585
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 22:24:18 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:18.179503 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.163933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:50283 (local address 127.1.74.190:44351)
0619 22:24:18.164126 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.164131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.165293 (+ 1162us) server_negotiation.cc:407] Connection header received
0619 22:24:18.166345 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.166348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.166418 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.166526 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:18.168255 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.168912 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.170135 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.170399 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.170518 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.174237 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:18.174241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:18.174244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:18.174279 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:18.176239 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.176981 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.176985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.176987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.177081 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.177467 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.177472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.177474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.177679 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:18.177783 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.178952 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.179227 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 22:24:18.180840 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.164350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:50283)
0619 22:24:18.165108 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.165152 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:18.166117 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:18.166748 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:18.166762 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:18.167222 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:18.168030 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.168056 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.169057 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.169060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.170001 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.170010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.170607 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.170611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.170790 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.171673 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:18.171699 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:18.173858 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:18.176425 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:18.176433 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:18.176461 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:18.176858 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:18.177196 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:18.177200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:18.177202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:18.177367 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:18.177805 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:18.177813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:18.178405 (+ 592us) client_negotiation.cc:769] Sending connection context
0619 22:24:18.178817 (+ 412us) client_negotiation.cc:240] Negotiation successful
0619 22:24:18.179402 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":145,"threads_started":1}
I20260619 22:24:18.182569 17585 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44351
I20260619 22:24:18.183101 17585 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:18.183761 17585 heartbeater.cc:511] Master 127.1.74.190:44351 requested a full tablet report, sending...
I20260619 22:24:18.185631 17259 ts_manager.cc:194] Registered new tserver with Master: be96f307cdc6428cb457c89ce4a29b55 (127.1.74.130:45193)
I20260619 22:24:18.186652 17259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:50283
W20260619 22:24:18.293462 17588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:18.293813 17588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:18.293905 17588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:18.298152 17588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:18.298260 17588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:18.298362 17588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:24:18.304085 17588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:38557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=37585
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:18.305301 17588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:18.306479 17588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:18.316766 17596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:18.316741 17593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:18.316774 17594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:18.316895 17588 server_base.cc:1034] running on GCE node
I20260619 22:24:18.317461 17588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:18.318200 17588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:18.319391 17588 hybrid_clock.cc:648] HybridClock initialized: now 1781907858319376 us; error 38 us; skew 500 ppm
Jun 19 22:24:18 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:18.322335 17588 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:24:18.323359 17588 webserver.cc:460] Webserver started at http://127.1.74.131:37585/ using document root <none> and password file <none>
I20260619 22:24:18.323952 17588 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:18.324028 17588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:18.328181 17588 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260619 22:24:18.330508 17603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:18.331915 17588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:24:18.332058 17588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "f11462123b6c400ba3f1b62c7ce87ba5"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "38a2052aa204bb774cd62f8359341c35"
server_key_iv: "7f3c30078ee3c95fe53bab79bdb70646"
server_key_version: "encryptionkey@0"
I20260619 22:24:18.332630 17588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:18.353498 17588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:18.357034 17588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:18.357252 17588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:18.357872 17588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:18.358732 17588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:18.358799 17588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:18.358903 17588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:18.358944 17588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:18.368738 17588 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:38557
I20260619 22:24:18.368759 17716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:38557 every 8 connection(s)
I20260619 22:24:18.369771 17588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:24:18.375188 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17588
Jun 19 22:24:18 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907858, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:18.384832 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.375905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.1.74.190:44351)
0619 22:24:18.376124 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.376127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.376372 (+ 245us) server_negotiation.cc:407] Connection header received
0619 22:24:18.376530 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.376533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.376579 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.376684 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:18.377508 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.378226 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.379259 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.379526 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.379652 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.381055 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:18.381064 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:18.381066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:18.381097 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:18.383043 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.383539 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.383542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.383544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.383624 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.383935 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.383938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.383939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.384128 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:18.384329 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.384464 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.384596 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260619 22:24:18.385108 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.371720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:34229 (local address 127.1.74.190:44351)
0619 22:24:18.371865 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.371869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.372957 (+ 1088us) server_negotiation.cc:407] Connection header received
0619 22:24:18.373879 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.373883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.373941 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.374046 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:18.375325 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.375909 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.377107 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.377438 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.377539 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.380621 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:18.380625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:18.380627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:18.380649 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:18.382619 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.383286 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.383290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.383292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.383376 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.383844 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.383847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.383850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.384064 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:18.384138 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.384829 (+ 691us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.384959 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:24:18.386089 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.372092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:34229)
0619 22:24:18.372804 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.372845 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:18.373681 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:18.374224 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:18.374230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:18.374590 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:18.375175 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.375189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.376063 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.376067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.376990 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.377000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.377607 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.377611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.377821 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.378622 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:18.378641 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:18.380423 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:18.382757 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:18.382762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:18.382789 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:18.383170 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:18.383516 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:18.383522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:18.383524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:18.383731 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:18.384175 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:18.384184 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:18.384510 (+ 326us) client_negotiation.cc:769] Sending connection context
0619 22:24:18.384753 (+ 243us) client_negotiation.cc:240] Negotiation successful
0619 22:24:18.385111 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":113,"threads_started":1}
I20260619 22:24:18.387267 17717 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44351
I20260619 22:24:18.387495 17717 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:18.387976 17717 heartbeater.cc:511] Master 127.1.74.190:44351 requested a full tablet report, sending...
I20260619 22:24:18.388985 17259 ts_manager.cc:194] Registered new tserver with Master: f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557)
I20260619 22:24:18.389621 17259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:34229
I20260619 22:24:18.398356 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:24:18.403069 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.399089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.1.74.190:44351)
0619 22:24:18.399276 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.399281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.399419 (+ 138us) server_negotiation.cc:407] Connection header received
0619 22:24:18.399625 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.399628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.399735 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.399861 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:24:18.400108 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.400647 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.401573 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.401849 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.401966 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.402288 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:24:18.402441 (+ 153us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 22:24:18.402894 (+ 453us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33850: token expired
Metrics: {"server-negotiator.queue_time_us":78}
W20260619 22:24:18.403156 17720 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33850: token expired
W20260619 22:24:18.403649 17186 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.74.190:44351)
I20260619 22:24:18.413091 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.404099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.1.74.190:44351)
0619 22:24:18.404245 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.404251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.404271 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:24:18.404538 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.404543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.404611 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.404728 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:18.405427 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.406252 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.407333 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.407573 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.407697 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.409164 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:18.409168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:18.409171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:18.409213 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:18.411131 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.411656 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.411662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.411664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.411765 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:18.412069 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:18.412073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:18.412075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:18.412295 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:18.412488 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.412737 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.412882 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:24:18.416030 17259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:24:18.419512 17259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:24:18.428567 17381 tablet_service.cc:1467] Processing CreateTablet for tablet 70ede406594d47c5ba71b545584b8f98 (DEFAULT_TABLE table=test-table [id=0ec1a95f40284361bd12fd5517a43ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:18.429262 17381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ede406594d47c5ba71b545584b8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:18.432303 17339 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap starting.
I20260619 22:24:18.433277 17339 tablet_bootstrap.cc:654] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:18.433518 17454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.427428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45193 (local address 127.0.0.1:53774)
0619 22:24:18.427575 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.427594 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:18.427670 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:18.429106 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:18.429110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:18.429139 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:18.429258 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.429266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.430998 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.431001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.431908 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.431918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.433020 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.433024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.433126 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.433188 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 22:24:18.433269 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 22:24:18.433368 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":45,"spinlock_wait_cycles":1408}
I20260619 22:24:18.434085 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.427636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.1.74.130:45193)
0619 22:24:18.428611 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.428623 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.428641 (+ 18us) server_negotiation.cc:407] Connection header received
0619 22:24:18.428705 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.428710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.428910 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.429405 (+ 495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:18.429583 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.430888 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.432042 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.432881 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.432994 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.433165 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.433479 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.433765 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":138,"threads_started":1}
I20260619 22:24:18.434638 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.427894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:38557 (local address 127.0.0.1:42048)
0619 22:24:18.428179 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.428192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:18.428274 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:18.429077 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:18.429080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:18.429099 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:18.429326 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.429335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.430821 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.430825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.432205 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.432216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.434259 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.434263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.434356 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.434375 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 22:24:18.434417 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 22:24:18.434472 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":162,"thread_start_us":99,"threads_started":1}
I20260619 22:24:18.435041 17339 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: No bootstrap required, opened a new log
I20260619 22:24:18.435145 17339 ts_tablet_manager.cc:1397] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 22:24:18.435328 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.428066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.1.74.131:38557)
0619 22:24:18.428530 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.428543 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.428578 (+ 35us) server_negotiation.cc:407] Connection header received
0619 22:24:18.428667 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.428673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.428911 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.429084 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:18.429450 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.430702 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.432352 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.434052 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.434822 (+ 770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.435016 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.435072 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.435181 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":106,"threads_started":1}
I20260619 22:24:18.435693 17339 raft_consensus.cc:348] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.435823 17339 raft_consensus.cc:374] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:18.435870 17339 raft_consensus.cc:729] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fc1aa19ebd48ca97cdf37069f43f66, State: Initialized, Role: FOLLOWER
I20260619 22:24:18.435993 17339 consensus_queue.cc:260] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.436272 17339 ts_tablet_manager.cc:1428] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 22:24:18.437395 17519 tablet_service.cc:1467] Processing CreateTablet for tablet 70ede406594d47c5ba71b545584b8f98 (DEFAULT_TABLE table=test-table [id=0ec1a95f40284361bd12fd5517a43ae6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260619 22:24:18.438138 17448 tablet.cc:2369] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:24:18.438542 17519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ede406594d47c5ba71b545584b8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:18.438175 17651 tablet_service.cc:1467] Processing CreateTablet for tablet 70ede406594d47c5ba71b545584b8f98 (DEFAULT_TABLE table=test-table [id=0ec1a95f40284361bd12fd5517a43ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:18.439239 17651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ede406594d47c5ba71b545584b8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:18.446125 17730 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Bootstrap starting.
I20260619 22:24:18.446136 17731 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Bootstrap starting.
I20260619 22:24:18.448691 17731 tablet_bootstrap.cc:654] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:18.448786 17730 tablet_bootstrap.cc:654] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:18.449594 17730 log.cc:824] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:18.449802 17731 log.cc:824] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:18.452666 17730 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: No bootstrap required, opened a new log
I20260619 22:24:18.452666 17731 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: No bootstrap required, opened a new log
I20260619 22:24:18.453058 17730 ts_tablet_manager.cc:1397] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 22:24:18.453073 17731 ts_tablet_manager.cc:1397] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 22:24:18.458405 17731 raft_consensus.cc:348] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.458405 17730 raft_consensus.cc:348] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.458874 17730 raft_consensus.cc:374] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:18.458874 17731 raft_consensus.cc:374] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:18.458937 17731 raft_consensus.cc:729] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11462123b6c400ba3f1b62c7ce87ba5, State: Initialized, Role: FOLLOWER
I20260619 22:24:18.458937 17730 raft_consensus.cc:729] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be96f307cdc6428cb457c89ce4a29b55, State: Initialized, Role: FOLLOWER
I20260619 22:24:18.459504 17730 consensus_queue.cc:260] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.459504 17731 consensus_queue.cc:260] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.460592 17717 heartbeater.cc:503] Master 127.1.74.190:44351 was elected leader, sending a full tablet report...
I20260619 22:24:18.460713 17731 ts_tablet_manager.cc:1428] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Time spent starting tablet: real 0.008s user 0.006s sys 0.003s
I20260619 22:24:18.460713 17730 ts_tablet_manager.cc:1428] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Time spent starting tablet: real 0.008s user 0.006s sys 0.003s
I20260619 22:24:18.460947 17585 heartbeater.cc:503] Master 127.1.74.190:44351 was elected leader, sending a full tablet report...
I20260619 22:24:18.566082 17735 raft_consensus.cc:482] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:24:18.566303 17735 raft_consensus.cc:504] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.567541 17735 leader_election.cc:288] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be96f307cdc6428cb457c89ce4a29b55 (127.1.74.130:45193), 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705)
I20260619 22:24:18.572867 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.567976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:57889 (local address 127.1.74.129:46705)
0619 22:24:18.568275 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.568282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.568303 (+ 21us) server_negotiation.cc:407] Connection header received
0619 22:24:18.568373 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.568376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.568450 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.568549 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:18.568888 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.569712 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.571599 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.572292 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.572400 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.572535 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.572639 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.572724 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":107,"threads_started":1}
I20260619 22:24:18.572865 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.567941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:44999 (local address 127.1.74.130:45193)
0619 22:24:18.568096 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.568101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.568620 (+ 519us) server_negotiation.cc:407] Connection header received
0619 22:24:18.568744 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.568746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.568786 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.568871 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:18.569094 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.569803 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.571599 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.572201 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.572340 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.572403 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.572639 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.572728 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:24:18.572916 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.567848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.1.74.131:57889)
0619 22:24:18.567989 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.568007 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:18.568113 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:18.568594 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:18.568598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:18.568636 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:18.568781 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.568786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.569899 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.569905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.571444 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.571457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.572392 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.572395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.572528 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.572545 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 22:24:18.572612 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 22:24:18.572715 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":189,"spinlock_wait_cycles":30208}
I20260619 22:24:18.572924 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.567855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45193 (local address 127.1.74.131:44999)
0619 22:24:18.568478 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.568489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:18.568643 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:18.568888 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:18.568892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:18.568913 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:18.568996 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.569000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.569915 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.569917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.571444 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:18.571457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.572356 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.572362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:18.572477 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.572524 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 22:24:18.572610 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 22:24:18.572710 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":408,"spinlock_wait_cycles":6144,"thread_start_us":121,"threads_started":1}
I20260619 22:24:18.573685 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ede406594d47c5ba71b545584b8f98" candidate_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" is_pre_election: true
I20260619 22:24:18.573827 17539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ede406594d47c5ba71b545584b8f98" candidate_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be96f307cdc6428cb457c89ce4a29b55" is_pre_election: true
I20260619 22:24:18.574054 17401 raft_consensus.cc:2454] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f11462123b6c400ba3f1b62c7ce87ba5 in term 0.
I20260619 22:24:18.574153 17539 raft_consensus.cc:2454] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f11462123b6c400ba3f1b62c7ce87ba5 in term 0.
I20260619 22:24:18.574680 17604 leader_election.cc:302] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 56fc1aa19ebd48ca97cdf37069f43f66, be96f307cdc6428cb457c89ce4a29b55, f11462123b6c400ba3f1b62c7ce87ba5; no voters:
I20260619 22:24:18.575012 17735 raft_consensus.cc:2781] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:24:18.575104 17735 raft_consensus.cc:482] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:24:18.575152 17735 raft_consensus.cc:3037] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:18.576231 17735 raft_consensus.cc:504] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.576606 17735 leader_election.cc:288] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [CANDIDATE]: Term 1 election: Requested vote from peers be96f307cdc6428cb457c89ce4a29b55 (127.1.74.130:45193), 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705)
I20260619 22:24:18.577070 17539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ede406594d47c5ba71b545584b8f98" candidate_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be96f307cdc6428cb457c89ce4a29b55"
I20260619 22:24:18.577241 17539 raft_consensus.cc:3037] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:18.577231 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ede406594d47c5ba71b545584b8f98" candidate_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
I20260619 22:24:18.577387 17401 raft_consensus.cc:3037] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:18.578392 17401 raft_consensus.cc:2454] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f11462123b6c400ba3f1b62c7ce87ba5 in term 1.
I20260619 22:24:18.578397 17539 raft_consensus.cc:2454] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f11462123b6c400ba3f1b62c7ce87ba5 in term 1.
I20260619 22:24:18.578785 17604 leader_election.cc:302] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56fc1aa19ebd48ca97cdf37069f43f66, f11462123b6c400ba3f1b62c7ce87ba5; no voters:
I20260619 22:24:18.579043 17735 raft_consensus.cc:2781] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:24:18.579298 17735 raft_consensus.cc:686] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 LEADER]: Becoming Leader. State: Replica: f11462123b6c400ba3f1b62c7ce87ba5, State: Running, Role: LEADER
I20260619 22:24:18.579640 17735 consensus_queue.cc:237] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:18.584403 17258 catalog_manager.cc:5495] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131). New cstate: current_term: 1 leader_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } health_report { overall_health: UNKNOWN } } }
W20260619 22:24:18.620777 17718 tablet.cc:2369] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:24:18.632336 17728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.628598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.1.74.131:38557)
0619 22:24:18.628756 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:18.628760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:18.628776 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:24:18.628906 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:18.628912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:18.628982 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:18.629109 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:24:18.629380 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.629911 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.630771 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:18.631044 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:18.631156 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:18.631483 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:24:18.631938 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:24:18.632085 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:18.632121 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 22:24:18.632185 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:24:18.639158 17401 raft_consensus.cc:1261] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Refusing update from remote peer f11462123b6c400ba3f1b62c7ce87ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:24:18.639168 17539 raft_consensus.cc:1261] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Refusing update from remote peer f11462123b6c400ba3f1b62c7ce87ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:24:18.639930 17738 consensus_queue.cc:1035] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:24:18.640177 17735 consensus_queue.cc:1035] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:24:18.648211 17741 mvcc.cc:205] Tried to move back new op lower bound from 7298694588977463296 to 7298694588745781248. Current Snapshot: MvccSnapshot[applied={T|T < 7298694588977463296 or (T in {7298694588977463296})}]
I20260619 22:24:18.649381 17743 mvcc.cc:205] Tried to move back new op lower bound from 7298694588977463296 to 7298694588745781248. Current Snapshot: MvccSnapshot[applied={T|T < 7298694588977463296}]
I20260619 22:24:18.656281 17259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:24:18.656450 17259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:24:18.658928 17259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Sending DeleteTablet for 3 replicas of tablet 70ede406594d47c5ba71b545584b8f98
I20260619 22:24:18.659804 17381 tablet_service.cc:1514] Processing DeleteTablet for tablet 70ede406594d47c5ba71b545584b8f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:18 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:48914
I20260619 22:24:18.659797 17651 tablet_service.cc:1514] Processing DeleteTablet for tablet 70ede406594d47c5ba71b545584b8f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:18 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:42048
I20260619 22:24:18.659814 17519 tablet_service.cc:1514] Processing DeleteTablet for tablet 70ede406594d47c5ba71b545584b8f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:18 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:53774
I20260619 22:24:18.660317 17751 tablet_replica.cc:330] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: stopping tablet replica
I20260619 22:24:18.660689 17751 raft_consensus.cc:2229] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:24:18.660790 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17297
I20260619 22:24:18.660907 17750 tablet_replica.cc:330] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: stopping tablet replica
I20260619 22:24:18.663108 17751 raft_consensus.cc:2258] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:18.664861 17751 ts_tablet_manager.cc:1905] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:24:18.667167 17752 tablet_replica.cc:330] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: stopping tablet replica
I20260619 22:24:18.667359 17751 ts_tablet_manager.cc:1918] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:24:18.667464 17751 log.cc:1196] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/70ede406594d47c5ba71b545584b8f98
I20260619 22:24:18.667830 17751 ts_tablet_manager.cc:1939] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Deleting consensus metadata
I20260619 22:24:18.667815 17752 raft_consensus.cc:2229] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 22:24:18.668202 17243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:46705 (error 108)
I20260619 22:24:18.668366 17752 raft_consensus.cc:2258] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:18.669431 17245 catalog_manager.cc:4841] TS f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557): tablet 70ede406594d47c5ba71b545584b8f98 (table test-table [id=0ec1a95f40284361bd12fd5517a43ae6]) successfully deleted
I20260619 22:24:18.669603 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17451
I20260619 22:24:18.671876 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.671537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.0.0.1:48928)
0619 22:24:18.671700 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.671782 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:46705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 22:24:18.672084 17243 catalog_manager.cc:4584] TS 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70ede406594d47c5ba71b545584b8f98: Network error: Client connection negotiation failed: client connection to 127.1.74.129:46705: connect: Connection refused (error 111)
I20260619 22:24:18.676118 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17588
I20260619 22:24:18.677428 17727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:18.676400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45193 (local address 127.0.0.1:53784)
0619 22:24:18.677277 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:18.677340 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:45193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":775}
I20260619 22:24:18.682389 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17226
I20260619 22:24:38.689320 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44351
--webserver_interface=127.1.74.190
--webserver_port=40313
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:44351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:38.815212 17754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:38.815537 17754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:38.815582 17754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:38.820019 17754 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 22:24:38.820096 17754 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 22:24:38.820113 17754 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 22:24:38.820127 17754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:24:38.820147 17754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:38.820164 17754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:38.820181 17754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:24:38.820196 17754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:24:38.825673 17754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:44351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=40313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:38.826722 17754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:38.827836 17754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:38.836544 17759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:38.836588 17760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:38.836544 17762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:38.836640 17754 server_base.cc:1034] running on GCE node
I20260619 22:24:38.837349 17754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:38.838464 17754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:38.839632 17754 hybrid_clock.cc:648] HybridClock initialized: now 1781907878839618 us; error 26 us; skew 500 ppm
Jun 19 22:24:38 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:38.842315 17754 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:24:38.843228 17754 webserver.cc:460] Webserver started at http://127.1.74.190:40313/ using document root <none> and password file <none>
I20260619 22:24:38.843753 17754 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:38.843812 17754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:38.847725 17754 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:24:38.849900 17769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:38.851214 17754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:24:38.851320 17754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "9877f97758634c95bbc4bc5eab2bcd2f"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "634701282f8c8f6ca29ee2533217e560"
server_key_iv: "8fe6b88adff3dfe42944bbaf9aa5a241"
server_key_version: "encryptionkey@0"
I20260619 22:24:38.851848 17754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:38.869068 17754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:38.872617 17754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:38.872808 17754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:38.880522 17754 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:44351
I20260619 22:24:38.880824 17821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:44351 every 8 connection(s)
I20260619 22:24:38.881709 17754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:24:38.886216 17822 sys_catalog.cc:263] Verifying existing consensus state
I20260619 22:24:38.887982 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17754
I20260619 22:24:38.888064 17822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap starting.
I20260619 22:24:38.888484 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:46705
--local_ip_for_outbound_sockets=127.1.74.129
--tserver_master_addrs=127.1.74.190:44351
--webserver_port=43107
--webserver_interface=127.1.74.129
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:24:38.911803 17822 log.cc:824] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:38.922953 17822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:24:38.923345 17822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Bootstrap complete.
I20260619 22:24:38.927490 17822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:38.927863 17822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9877f97758634c95bbc4bc5eab2bcd2f, State: Initialized, Role: FOLLOWER
I20260619 22:24:38.928380 17822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:38.928505 17822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:24:38.928592 17822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:24:38.928706 17822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 2 FOLLOWER]: Advancing to term 3
I20260619 22:24:38.930619 17822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:38.930986 17822 leader_election.cc:302] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9877f97758634c95bbc4bc5eab2bcd2f; no voters:
I20260619 22:24:38.931331 17822 leader_election.cc:288] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 22:24:38.931416 17826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 22:24:38.931612 17826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [term 3 LEADER]: Becoming Leader. State: Replica: 9877f97758634c95bbc4bc5eab2bcd2f, State: Running, Role: LEADER
I20260619 22:24:38.932035 17826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } }
I20260619 22:24:38.932605 17822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:24:38.933055 17826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9877f97758634c95bbc4bc5eab2bcd2f. Latest consensus state: current_term: 3 leader_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } } }
I20260619 22:24:38.933169 17826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:38.933549 17830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:24:38.934041 17827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9877f97758634c95bbc4bc5eab2bcd2f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44351 } } }
I20260619 22:24:38.934247 17827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:38.938956 17830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:24:38.940032 17830 catalog_manager.cc:1259] Loaded cluster ID: 9eaa5a9eb65349d3a8572022a1fabecd
I20260619 22:24:38.940104 17830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:24:38.942731 17830 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:24:38.943606 17830 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Loaded TSK: 0
I20260619 22:24:38.943704 17830 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9877f97758634c95bbc4bc5eab2bcd2f: Loaded TSK: 1
F20260619 22:24:38.944522 17830 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f935d28e9ad google::LogMessage::Fail() at ??:0
@ 0x7f935d29188a google::LogMessage::SendToLog() at ??:0
@ 0x7f935d28e4de google::LogMessage::Flush() at ??:0
@ 0x7f935d28fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f935ff5218d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f935ff4af0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9367bf77bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9367bd199c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9367bd2178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9367c13df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9367c3282b std::function<>::operator()() at ??:0
@ 0x7f9367bd1b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9367bd2f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9367bcf579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9367c13640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f935f4e03d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f935f4e0c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f935f4e23f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f935f4d1512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9360b156db start_thread at ??:0
@ 0x7f935bd8c61f clone at ??:0
W20260619 22:24:39.059720 17824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:39.060039 17824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:39.060091 17824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:39.064518 17824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:39.064605 17824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:39.064669 17824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:24:39.070786 17824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:46705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=43107
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:39.072320 17824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:39.073410 17824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:39.083940 17869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:39.083949 17870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:39.083943 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:39.084412 17824 server_base.cc:1034] running on GCE node
I20260619 22:24:39.084951 17824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:39.085767 17824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:39.087347 17824 hybrid_clock.cc:648] HybridClock initialized: now 1781907879087291 us; error 61 us; skew 500 ppm
Jun 19 22:24:39 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907879, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:39.092254 17824 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:24:39.093547 17824 webserver.cc:460] Webserver started at http://127.1.74.129:43107/ using document root <none> and password file <none>
I20260619 22:24:39.094370 17824 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:39.094491 17824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:39.099442 17824 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:24:39.101915 17879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:39.103458 17824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:24:39.103574 17824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "3cc8e55042e89af0beffb398a7b31063"
server_key_iv: "9f2c55e26c52a33df4cb95e67e8ce258"
server_key_version: "encryptionkey@0"
I20260619 22:24:39.104204 17824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:39.136571 17824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:39.140173 17824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:39.140398 17824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:39.141034 17824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:39.142179 17886 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 22:24:39.150081 17824 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 22:24:39.150290 17824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.000s
I20260619 22:24:39.150399 17824 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 22:24:39.153798 17824 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 22:24:39.153919 17824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260619 22:24:39.154497 17886 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap starting.
I20260619 22:24:39.165942 17824 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:46705
I20260619 22:24:39.166299 17993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:46705 every 8 connection(s)
I20260619 22:24:39.167333 17824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:24:39.167920 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17824
I20260619 22:24:39.168474 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:45193
--local_ip_for_outbound_sockets=127.1.74.130
--tserver_master_addrs=127.1.74.190:44351
--webserver_port=40115
--webserver_interface=127.1.74.130
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:24:39.180469 17886 log.cc:824] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:39.190508 17886 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:24:39.191037 17886 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Bootstrap complete.
I20260619 22:24:39.192559 17886 ts_tablet_manager.cc:1397] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent bootstrapping tablet: real 0.039s user 0.021s sys 0.014s
I20260619 22:24:39.196074 17886 raft_consensus.cc:348] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:39.196699 17886 raft_consensus.cc:729] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fc1aa19ebd48ca97cdf37069f43f66, State: Initialized, Role: FOLLOWER
I20260619 22:24:39.197352 17886 consensus_queue.cc:260] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:39.198570 17886 ts_tablet_manager.cc:1428] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260619 22:24:39.209360 17997 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:24:39.169846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:51931)
0619 22:24:39.172671 (+ 2825us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.172729 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:39.173884 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:39.208354 (+ 34470us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2479,"thread_start_us":280,"threads_started":1}
W20260619 22:24:39.210007 17994 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:44351 (0 consecutive failures): Network error: Failed to ping master at 127.1.74.190:44351: Client connection negotiation failed: client connection to 127.1.74.190:44351: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 22:24:39.210968 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.210601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:42089)
0619 22:24:39.210751 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.210874 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 22:24:39.211843 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.211590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:46987)
0619 22:24:39.211718 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.211780 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 22:24:39.212103 17994 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 22:24:39.300323 17996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:39.300647 17996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:39.300747 17996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:39.305296 17996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:39.305378 17996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:39.305452 17996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:24:39.311878 17996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:45193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=40115
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:39.313217 17996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:39.314383 17996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:39.324718 18005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:39.324694 18007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:39.324811 17996 server_base.cc:1034] running on GCE node
W20260619 22:24:39.324692 18004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:39.325446 17996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:39.326206 17996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:39.327409 17996 hybrid_clock.cc:648] HybridClock initialized: now 1781907879327350 us; error 74 us; skew 500 ppm
Jun 19 22:24:39 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907879, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:39.331200 17996 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:24:39.332285 17996 webserver.cc:460] Webserver started at http://127.1.74.130:40115/ using document root <none> and password file <none>
I20260619 22:24:39.332918 17996 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:39.333029 17996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:39.337517 17996 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:24:39.340219 18014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:39.341805 17996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:24:39.341996 17996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "be96f307cdc6428cb457c89ce4a29b55"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "57f57e96dd267ccbc89319cfafb4f7ae"
server_key_iv: "d96c13be278d5db0fc692749dfde18bd"
server_key_version: "encryptionkey@0"
I20260619 22:24:39.342618 17996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:39.354600 17996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:39.359182 17996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:39.359467 17996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:39.360209 17996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:39.361565 18021 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 22:24:39.366050 17996 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 22:24:39.366194 17996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260619 22:24:39.366285 17996 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 22:24:39.368291 17996 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 22:24:39.368371 17996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260619 22:24:39.368690 18021 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Bootstrap starting.
I20260619 22:24:39.374393 18021 log.cc:824] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:39.379779 17996 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:45193
I20260619 22:24:39.379828 18129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:45193 every 8 connection(s)
I20260619 22:24:39.380673 17996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:24:39.383065 18021 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:24:39.384049 18021 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Bootstrap complete.
I20260619 22:24:39.384136 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.382645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:54119)
0619 22:24:39.383208 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.383352 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":171,"threads_started":1}
I20260619 22:24:39.384480 18021 ts_tablet_manager.cc:1397] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
W20260619 22:24:39.384603 18130 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:44351 (0 consecutive failures): Network error: Failed to ping master at 127.1.74.190:44351: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
I20260619 22:24:39.385267 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.385026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:59963)
0619 22:24:39.385153 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.385212 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 22:24:39.385908 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.385690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:53929)
0619 22:24:39.385796 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.385842 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 22:24:39.386073 18130 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 22:24:39.387701 18021 raft_consensus.cc:348] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:39.388226 18021 raft_consensus.cc:729] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: be96f307cdc6428cb457c89ce4a29b55, State: Initialized, Role: FOLLOWER
I20260619 22:24:39.388692 18021 consensus_queue.cc:260] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:39.389636 18021 ts_tablet_manager.cc:1428] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:24:39.389853 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.389577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:42875)
0619 22:24:39.389703 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.389782 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 22:24:39.390460 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.390247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:43245)
0619 22:24:39.390354 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.390400 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 22:24:39.390900 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 17996
I20260619 22:24:39.391346 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:38557
--local_ip_for_outbound_sockets=127.1.74.131
--tserver_master_addrs=127.1.74.190:44351
--webserver_port=37585
--webserver_interface=127.1.74.131
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:39.508276 18134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:39.508584 18134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:39.508657 18134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:39.512971 18134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:39.513031 18134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:39.513089 18134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:24:39.518400 18134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:38557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=37585
--tserver_master_addrs=127.1.74.190:44351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:39.519472 18134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:39.520488 18134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:39.530197 18142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:39.530170 18140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:39.530175 18139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:39.530488 18134 server_base.cc:1034] running on GCE node
I20260619 22:24:39.530967 18134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:39.531658 18134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:39.532852 18134 hybrid_clock.cc:648] HybridClock initialized: now 1781907879532835 us; error 27 us; skew 500 ppm
Jun 19 22:24:39 dist-test-slave-k84h krb5kdc[16678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907879, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:39.535524 18134 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:24:39.536334 18134 webserver.cc:460] Webserver started at http://127.1.74.131:37585/ using document root <none> and password file <none>
I20260619 22:24:39.536865 18134 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:39.536923 18134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:39.540830 18134 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:24:39.542964 18149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:39.544269 18134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:24:39.544373 18134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "f11462123b6c400ba3f1b62c7ce87ba5"
format_stamp: "Formatted at 2026-06-19 22:23:56 on dist-test-slave-k84h"
server_key: "38a2052aa204bb774cd62f8359341c35"
server_key_iv: "7f3c30078ee3c95fe53bab79bdb70646"
server_key_version: "encryptionkey@0"
I20260619 22:24:39.544903 18134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:39.581842 18134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:39.585237 18134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:39.585410 18134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:39.585978 18134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:39.586791 18134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:39.586907 18134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:39.586980 18134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:39.587004 18134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:39.597076 18134 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:38557
I20260619 22:24:39.597095 18262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:38557 every 8 connection(s)
I20260619 22:24:39.598193 18134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:24:39.599435 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18134
I20260619 22:24:39.602912 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.601173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:42407)
0619 22:24:39.601958 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.602074 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":158,"threads_started":1}
W20260619 22:24:39.603551 18263 heartbeater.cc:644] Failed to heartbeat to 127.1.74.190:44351 (0 consecutive failures): Network error: Failed to ping master at 127.1.74.190:44351: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
I20260619 22:24:39.604699 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.604135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:38081)
0619 22:24:39.604509 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.604590 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
I20260619 22:24:39.605893 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:39.605454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:47265)
0619 22:24:39.605679 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:39.605779 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260619 22:24:39.606091 18263 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 22:24:40.213318 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.212816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:54457)
0619 22:24:40.213124 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:40.213204 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":78,"threads_started":1}
I20260619 22:24:40.391921 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.391403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:43733)
0619 22:24:40.391751 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:40.391821 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":83,"threads_started":1}
I20260619 22:24:40.548815 18269 raft_consensus.cc:482] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:24:40.549065 18269 raft_consensus.cc:504] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:40.550209 18269 leader_election.cc:288] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557), 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705)
Jun 19 22:24:40 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907879, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.129@KRBTEST.COM
Jun 19 22:24:40 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907879, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.131@KRBTEST.COM
I20260619 22:24:40.563797 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.550832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:55917 (local address 127.1.74.129:46705)
0619 22:24:40.551184 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:40.551191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:40.551218 (+ 27us) server_negotiation.cc:407] Connection header received
0619 22:24:40.551606 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:40.551612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:40.551818 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:40.551960 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:40.553348 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.554290 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.555653 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.556114 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.556297 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:40.559056 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:40.559063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:40.559075 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:40.559104 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:40.560857 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:40.561437 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:40.561444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:40.561451 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:40.561628 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:40.562026 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:40.562030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:40.562031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:40.562529 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:40.562632 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:40.563345 (+ 713us) server_negotiation.cc:299] Negotiation successful
0619 22:24:40.563609 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260619 22:24:40.563798 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.550638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:48509 (local address 127.1.74.131:38557)
0619 22:24:40.551077 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:40.551090 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:40.551133 (+ 43us) server_negotiation.cc:407] Connection header received
0619 22:24:40.552164 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:40.552170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:40.552524 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:40.552802 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:40.553344 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.554293 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.555655 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.556115 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.556276 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:40.559402 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:40.559411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:40.559425 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:40.559462 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:40.561296 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:40.561857 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:40.561863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:40.561867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:40.562001 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:40.562273 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:40.562275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:40.562276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:40.562675 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:40.562798 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:40.563343 (+ 545us) server_negotiation.cc:299] Negotiation successful
0619 22:24:40.563590 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":122,"threads_started":1}
I20260619 22:24:40.563788 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.550446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:46705 (local address 127.1.74.130:55917)
0619 22:24:40.550752 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:40.550767 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:40.551272 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:40.552090 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:40.552100 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:40.552607 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:40.553178 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:40.553192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.554455 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.554459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:40.555529 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:40.555538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.556267 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.556271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:40.556497 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:40.557142 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:40.557158 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:40.558827 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:40.561028 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:40.561033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:40.561046 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:40.561311 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:40.561737 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:40.561742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:40.561747 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:40.561923 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:40.562649 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:40.562656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:40.563010 (+ 354us) client_negotiation.cc:769] Sending connection context
0619 22:24:40.563212 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 22:24:40.563586 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":260,"spinlock_wait_cycles":13312,"thread_start_us":85,"threads_started":1}
I20260619 22:24:40.563903 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.550405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:38557 (local address 127.1.74.130:48509)
0619 22:24:40.550586 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:40.550616 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:40.551280 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:40.552763 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:40.552766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:40.552928 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:40.553176 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:40.553191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.554453 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.554458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:40.555527 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:40.555537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:40.556241 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:40.556245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:40.556477 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:40.557190 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:40.557202 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:40.559186 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:40.561426 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:40.561429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:40.561431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:40.561645 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:40.562090 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:40.562093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:40.562095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:40.562175 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:40.562798 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:40.562801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:40.563001 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 22:24:40.563213 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 22:24:40.563684 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":231,"spinlock_wait_cycles":13056}
I20260619 22:24:40.565515 17948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ede406594d47c5ba71b545584b8f98" candidate_uuid: "be96f307cdc6428cb457c89ce4a29b55" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" is_pre_election: true
I20260619 22:24:40.565900 17948 raft_consensus.cc:2454] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be96f307cdc6428cb457c89ce4a29b55 in term 1.
W20260619 22:24:40.566991 18017 leader_election.cc:341] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557): Not found: Tablet not found: 70ede406594d47c5ba71b545584b8f98
I20260619 22:24:40.567102 18017 leader_election.cc:302] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56fc1aa19ebd48ca97cdf37069f43f66, be96f307cdc6428cb457c89ce4a29b55; no voters: f11462123b6c400ba3f1b62c7ce87ba5
I20260619 22:24:40.567384 18269 raft_consensus.cc:2781] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 22:24:40.567480 18269 raft_consensus.cc:482] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:24:40.567538 18269 raft_consensus.cc:3037] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 1 FOLLOWER]: Advancing to term 2
I20260619 22:24:40.569664 18269 raft_consensus.cc:504] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:40.570202 18269 leader_election.cc:288] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 2 election: Requested vote from peers f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557), 56fc1aa19ebd48ca97cdf37069f43f66 (127.1.74.129:46705)
I20260619 22:24:40.570725 17948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70ede406594d47c5ba71b545584b8f98" candidate_uuid: "be96f307cdc6428cb457c89ce4a29b55" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "56fc1aa19ebd48ca97cdf37069f43f66"
I20260619 22:24:40.570901 17948 raft_consensus.cc:3037] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 1 FOLLOWER]: Advancing to term 2
W20260619 22:24:40.570930 18017 leader_election.cc:341] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557): Not found: Tablet not found: 70ede406594d47c5ba71b545584b8f98
I20260619 22:24:40.573264 17948 raft_consensus.cc:2454] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be96f307cdc6428cb457c89ce4a29b55 in term 2.
I20260619 22:24:40.573635 18015 leader_election.cc:302] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56fc1aa19ebd48ca97cdf37069f43f66, be96f307cdc6428cb457c89ce4a29b55; no voters: f11462123b6c400ba3f1b62c7ce87ba5
I20260619 22:24:40.573810 18269 raft_consensus.cc:2781] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 22:24:40.574219 18269 raft_consensus.cc:686] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [term 2 LEADER]: Becoming Leader. State: Replica: be96f307cdc6428cb457c89ce4a29b55, State: Running, Role: LEADER
I20260619 22:24:40.574988 18269 consensus_queue.cc:237] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:40.575140 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.574889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:36137)
0619 22:24:40.575000 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:40.575054 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 22:24:40.607620 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:40.606916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:58885)
0619 22:24:40.607395 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:40.607474 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":134,"threads_started":1}
W20260619 22:24:40.981791 18017 consensus_peers.cc:487] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 -> Peer f11462123b6c400ba3f1b62c7ce87ba5 (127.1.74.131:38557): Couldn't send request to peer f11462123b6c400ba3f1b62c7ce87ba5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70ede406594d47c5ba71b545584b8f98. This is attempt 1: this message will repeat every 5th retry.
I20260619 22:24:41.189014 17948 raft_consensus.cc:1261] T 70ede406594d47c5ba71b545584b8f98 P 56fc1aa19ebd48ca97cdf37069f43f66 [term 2 FOLLOWER]: Refusing update from remote peer be96f307cdc6428cb457c89ce4a29b55: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260619 22:24:41.190248 18269 consensus_queue.cc:1035] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 22:24:41.193133 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.192711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:53817)
0619 22:24:41.192993 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:41.193040 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":75,"threads_started":1}
I20260619 22:24:41.196410 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.196035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:55179)
0619 22:24:41.196241 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:41.196301 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260619 22:24:41.196588 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.196319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:59609)
0619 22:24:41.196423 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:41.196476 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 22:24:41.461879 18283 ts_tablet_manager.cc:927] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Initiating tablet copy from peer be96f307cdc6428cb457c89ce4a29b55 (127.1.74.130:45193)
I20260619 22:24:41.462515 18283 tablet_copy_client.cc:316] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: tablet copy: Beginning tablet copy session from remote peer at address 127.1.74.130:45193
Jun 19 22:24:41 dist-test-slave-k84h krb5kdc[16678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907879, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.130@KRBTEST.COM
I20260619 22:24:41.480165 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.463090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:45193 (local address 127.1.74.131:54189)
0619 22:24:41.463446 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:41.463471 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:41.463623 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:41.464577 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:41.464583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:41.465005 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:41.465144 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:41.465151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:41.466541 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:41.466545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:41.467427 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:41.467440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:41.468353 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:41.468357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:41.468481 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:41.473349 (+ 4868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:41.473375 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:41.475282 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:41.477656 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:41.477667 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:41.477673 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:41.477951 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:41.478684 (+ 733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:41.478687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:41.478689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:41.478785 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:41.479550 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:41.479557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:41.479698 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 22:24:41.479829 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 22:24:41.479995 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":85,"threads_started":1}
I20260619 22:24:41.480358 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.463416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:54189 (local address 127.1.74.130:45193)
0619 22:24:41.463919 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:41.463932 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:41.463970 (+ 38us) server_negotiation.cc:407] Connection header received
0619 22:24:41.464043 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:41.464050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:41.464318 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:41.464513 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:41.465331 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:41.466340 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:41.467603 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:41.468107 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:41.468257 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:41.475554 (+ 7297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:41.475568 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:41.475583 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:41.475620 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:41.477485 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:41.478178 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:41.478186 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:41.478188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:41.478476 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:41.479072 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:41.479076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:41.479077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:41.479405 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:41.479541 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:41.480006 (+ 465us) server_negotiation.cc:299] Negotiation successful
0619 22:24:41.480170 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":125,"threads_started":1}
I20260619 22:24:41.481441 18104 tablet_copy_service.cc:140] P be96f307cdc6428cb457c89ce4a29b55: Received BeginTabletCopySession request for tablet 70ede406594d47c5ba71b545584b8f98 from peer f11462123b6c400ba3f1b62c7ce87ba5 ({username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:54189)
I20260619 22:24:41.481611 18104 tablet_copy_service.cc:161] P be96f307cdc6428cb457c89ce4a29b55: Beginning new tablet copy session on tablet 70ede406594d47c5ba71b545584b8f98 from peer f11462123b6c400ba3f1b62c7ce87ba5 at {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:54189: session id = f11462123b6c400ba3f1b62c7ce87ba5-70ede406594d47c5ba71b545584b8f98
I20260619 22:24:41.483129 18104 tablet_copy_source_session.cc:205] T 70ede406594d47c5ba71b545584b8f98 P be96f307cdc6428cb457c89ce4a29b55: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 22:24:41.489138 18283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70ede406594d47c5ba71b545584b8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:41.496336 18283 tablet_copy_client.cc:791] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: tablet copy: Starting download of 0 data blocks...
I20260619 22:24:41.496557 18283 tablet_copy_client.cc:655] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: tablet copy: Starting download of 1 WAL segments...
I20260619 22:24:41.498486 18283 tablet_copy_client.cc:531] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 22:24:41.500316 18283 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Bootstrap starting.
I20260619 22:24:41.505905 18283 log.cc:824] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:41.512876 18283 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 22:24:41.513247 18283 tablet_bootstrap.cc:492] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Bootstrap complete.
I20260619 22:24:41.513624 18283 ts_tablet_manager.cc:1397] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260619 22:24:41.516019 18283 raft_consensus.cc:348] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:41.516248 18283 raft_consensus.cc:729] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11462123b6c400ba3f1b62c7ce87ba5, State: Initialized, Role: FOLLOWER
I20260619 22:24:41.516702 18283 consensus_queue.cc:260] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be96f307cdc6428cb457c89ce4a29b55" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 45193 } } peers { permanent_uuid: "f11462123b6c400ba3f1b62c7ce87ba5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 38557 } } peers { permanent_uuid: "56fc1aa19ebd48ca97cdf37069f43f66" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 46705 } }
I20260619 22:24:41.517773 18283 ts_tablet_manager.cc:1428] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260619 22:24:41.517901 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.517612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:48165)
0619 22:24:41.517724 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:41.517772 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 22:24:41.518579 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:41.518352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:39073)
0619 22:24:41.518475 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:41.518528 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 22:24:41.518543 18104 tablet_copy_service.cc:342] P be96f307cdc6428cb457c89ce4a29b55: Request end of tablet copy session f11462123b6c400ba3f1b62c7ce87ba5-70ede406594d47c5ba71b545584b8f98 received from {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:54189
I20260619 22:24:41.518672 18104 tablet_copy_service.cc:433] P be96f307cdc6428cb457c89ce4a29b55: ending tablet copy session f11462123b6c400ba3f1b62c7ce87ba5-70ede406594d47c5ba71b545584b8f98 on tablet 70ede406594d47c5ba71b545584b8f98 with peer f11462123b6c400ba3f1b62c7ce87ba5
I20260619 22:24:42.072002 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:42.071525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:40061)
0619 22:24:42.071811 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:42.071875 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":61,"threads_started":1}
I20260619 22:24:42.071938 18217 raft_consensus.cc:1203] T 70ede406594d47c5ba71b545584b8f98 P f11462123b6c400ba3f1b62c7ce87ba5 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260619 22:24:42.073370 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:42.073025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:59213)
0619 22:24:42.073220 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:42.073276 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260619 22:24:42.198029 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:42.197524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:34515)
0619 22:24:42.197864 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:42.197924 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":99,"threads_started":1}
I20260619 22:24:43.073832 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:43.073180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:50071)
0619 22:24:43.073602 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:43.073702 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":108,"threads_started":1}
I20260619 22:24:43.074683 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:43.074272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:45835)
0619 22:24:43.074543 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:43.074596 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260619 22:24:43.199488 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:43.199059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:56745)
0619 22:24:43.199324 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:43.199388 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260619 22:24:44.075202 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:44.074688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:57777)
0619 22:24:44.074992 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:44.075067 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":89,"threads_started":1}
I20260619 22:24:44.075906 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:44.075510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:48157)
0619 22:24:44.075779 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:44.075830 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":59,"threads_started":1}
I20260619 22:24:44.201102 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:44.200568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:55577)
0619 22:24:44.200909 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:44.200992 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":87,"threads_started":1}
I20260619 22:24:45.076802 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:45.076316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:33431)
0619 22:24:45.076606 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:45.076690 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":77,"threads_started":1}
I20260619 22:24:45.077396 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:45.076907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:40239)
0619 22:24:45.077205 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:45.077290 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
I20260619 22:24:45.077685 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:45.077392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:47983)
0619 22:24:45.077560 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:45.077613 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 22:24:45.202672 18302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:45.202170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:34241)
0619 22:24:45.202493 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:45.202571 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":74,"threads_started":1}
I20260619 22:24:46.078771 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:46.078299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:34633)
0619 22:24:46.078600 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:46.078670 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
I20260619 22:24:46.078804 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:46.078396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:45539)
0619 22:24:46.078652 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:46.078721 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
I20260619 22:24:46.204156 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:46.203649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:39147)
0619 22:24:46.203971 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:46.204049 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":84,"threads_started":1}
I20260619 22:24:47.080214 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:47.079687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:58275)
0619 22:24:47.080027 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:47.080101 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
I20260619 22:24:47.080348 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:47.079913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:53399)
0619 22:24:47.080171 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:47.080233 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":71,"threads_started":1}
I20260619 22:24:47.205612 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:47.205114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:40915)
0619 22:24:47.205453 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:47.205514 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":122,"threads_started":1}
I20260619 22:24:48.081394 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:48.080973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:52153)
0619 22:24:48.081234 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:48.081299 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260619 22:24:48.081429 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:48.080972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:32781)
0619 22:24:48.081273 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:48.081334 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":120,"threads_started":1}
I20260619 22:24:48.206981 18311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:48.206463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:53413)
0619 22:24:48.206762 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:48.206821 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":96,"threads_started":1}
I20260619 22:24:49.082777 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:49.082311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:34447)
0619 22:24:49.082594 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:49.082674 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260619 22:24:49.083135 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:49.082403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:48639)
0619 22:24:49.082717 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:49.082782 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":82,"threads_started":1}
I20260619 22:24:49.208387 18314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:49.207904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:60773)
0619 22:24:49.208188 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:49.208260 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":74,"threads_started":1}
I20260619 22:24:50.084262 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:50.083705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:40839)
0619 22:24:50.084052 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:50.084115 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260619 22:24:50.084535 18316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:50.084099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:59185)
0619 22:24:50.084369 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:50.084435 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":87,"threads_started":1}
I20260619 22:24:50.209743 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:50.209335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:51985)
0619 22:24:50.209587 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:50.209645 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
I20260619 22:24:51.085759 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:51.085230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:35959)
0619 22:24:51.085580 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:51.085650 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":129,"threads_started":1}
I20260619 22:24:51.085801 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:51.085397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:59535)
0619 22:24:51.085646 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:51.085710 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":81,"threads_started":1}
I20260619 22:24:51.211323 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:51.210714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:34155)
0619 22:24:51.211122 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:51.211209 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":179,"threads_started":1}
I20260619 22:24:52.087374 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:52.086871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:37395)
0619 22:24:52.087233 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:52.087287 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":89,"threads_started":1}
I20260619 22:24:52.087364 18322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:52.086841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:39513)
0619 22:24:52.087171 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:52.087260 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":86,"threads_started":1}
I20260619 22:24:52.212532 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:52.212111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:44017)
0619 22:24:52.212373 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:52.212436 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":69,"threads_started":1}
I20260619 22:24:53.088619 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:53.088185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:39757)
0619 22:24:53.088448 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:53.088508 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260619 22:24:53.088620 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:53.088185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:60401)
0619 22:24:53.088448 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:53.088508 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
I20260619 22:24:53.213711 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:53.213288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:41673)
0619 22:24:53.213557 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:53.213616 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260619 22:24:54.089852 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:54.089415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.130:52037)
0619 22:24:54.089689 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:54.089756 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":92,"threads_started":1}
I20260619 22:24:54.089882 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:54.089415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.131:33905)
0619 22:24:54.089746 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:54.089798 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":95,"threads_started":1}
I20260619 22:24:54.215729 18329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:54.215080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44351 (local address 127.1.74.129:50279)
0619 22:24:54.215508 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:54.215595 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.190:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":107,"threads_started":1}
I20260619 22:24:54.607836 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17824
I20260619 22:24:54.614293 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 17996
I20260619 22:24:54.622573 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18134
2026-06-19T22:24:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59825 ms)
[----------] 30 tests from SecurityITest (251404 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:24:54 dist-test-slave-k84h krb5kdc[18332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:24:54 dist-test-slave-k84h krb5kdc[18332](info): set up 2 sockets
Jun 19 22:24:54 dist-test-slave-k84h krb5kdc[18332](info): commencing operation
krb5kdc: starting...
W20260619 22:24:55.711081 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:24:55 dist-test-slave-k84h krb5kdc[18332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:24:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:24:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:24:55.859145 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37415
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:55.977855 18345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:55.978220 18345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:55.978276 18345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:55.983681 18345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:24:55.983788 18345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:55.983810 18345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:55.983829 18345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:24:55.983846 18345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:24:55.990027 18345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:55.991300 18345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:55.992391 18345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:56.001945 18353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:56.001951 18351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.002028 18345 server_base.cc:1034] running on GCE node
W20260619 22:24:56.001950 18350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.002794 18345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:56.004129 18345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:56.005316 18345 hybrid_clock.cc:648] HybridClock initialized: now 1781907896005299 us; error 37 us; skew 500 ppm
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:56.008512 18345 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:24:56.009533 18345 webserver.cc:460] Webserver started at http://127.1.74.190:38229/ using document root <none> and password file <none>
I20260619 22:24:56.010143 18345 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:56.010243 18345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:56.010512 18345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:56.013044 18345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "4aea3038eb044837a868a352df6b3975"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "3ae74339f5c90aea92a4e0649b550add"
server_key_iv: "fc9b0488af9c90322193449c12283a5c"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.013680 18345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "4aea3038eb044837a868a352df6b3975"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "3ae74339f5c90aea92a4e0649b550add"
server_key_iv: "fc9b0488af9c90322193449c12283a5c"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.017809 18345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 22:24:56.020486 18360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.022136 18345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:24:56.022308 18345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "4aea3038eb044837a868a352df6b3975"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "3ae74339f5c90aea92a4e0649b550add"
server_key_iv: "fc9b0488af9c90322193449c12283a5c"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.022440 18345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:56.057771 18345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:56.061245 18345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:56.061475 18345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:56.069352 18345 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:37415
I20260619 22:24:56.069386 18412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:37415 every 8 connection(s)
I20260619 22:24:56.070456 18345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:24:56.073913 18413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:56.077140 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18345
I20260619 22:24:56.077312 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:24:56.077732 1322 external_mini_cluster.cc:1428] Setting key 10cd6913dfe320c0b88eca4eb17f20f7
I20260619 22:24:56.080335 18413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975: Bootstrap starting.
I20260619 22:24:56.083079 18413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:56.083868 18413 log.cc:824] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:56.086236 18413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975: No bootstrap required, opened a new log
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:56.090675 18413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aea3038eb044837a868a352df6b3975" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37415 } }
I20260619 22:24:56.090991 18413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:56.091069 18413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aea3038eb044837a868a352df6b3975, State: Initialized, Role: FOLLOWER
I20260619 22:24:56.091573 18413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aea3038eb044837a868a352df6b3975" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37415 } }
I20260619 22:24:56.091737 18413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:24:56.091828 18413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:24:56.091951 18413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:56.093266 18413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aea3038eb044837a868a352df6b3975" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37415 } }
I20260619 22:24:56.093643 18413 leader_election.cc:302] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aea3038eb044837a868a352df6b3975; no voters:
I20260619 22:24:56.093991 18413 leader_election.cc:288] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:24:56.094111 18418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:24:56.094378 18418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [term 1 LEADER]: Becoming Leader. State: Replica: 4aea3038eb044837a868a352df6b3975, State: Running, Role: LEADER
I20260619 22:24:56.094540 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.079449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56980 (local address 127.1.74.190:37415)
0619 22:24:56.079904 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.079926 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.079963 (+ 37us) server_negotiation.cc:407] Connection header received
0619 22:24:56.080721 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.080742 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.081090 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.081389 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:56.082632 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.084228 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.085409 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.085891 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.086046 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.089087 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:56.089092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:56.089108 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:56.089140 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:56.091202 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.091681 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.091687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.091693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.091847 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.092133 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.092139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.092144 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.092700 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:56.092928 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.093174 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.093500 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":153,"threads_started":1}
I20260619 22:24:56.094720 18418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aea3038eb044837a868a352df6b3975" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37415 } }
I20260619 22:24:56.095286 18413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:24:56.096022 18420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aea3038eb044837a868a352df6b3975. Latest consensus state: current_term: 1 leader_uuid: "4aea3038eb044837a868a352df6b3975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aea3038eb044837a868a352df6b3975" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37415 } } }
I20260619 22:24:56.096207 18420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:56.096392 18419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aea3038eb044837a868a352df6b3975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aea3038eb044837a868a352df6b3975" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37415 } } }
I20260619 22:24:56.096495 18419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975 [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:56.097550 18422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:24:56.100498 18422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:24:56.106997 18422 catalog_manager.cc:1347] Generated new cluster ID: e08cf8f0ae334d39ba855c5490b632cc
I20260619 22:24:56.107105 18422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:24:56.117602 18422 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:24:56.118732 18422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:24:56.126909 18422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975: Generated new TSK 0
I20260619 22:24:56.127810 18422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:24:56.163295 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37415
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:56.292011 18439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:56.292366 18439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:56.292450 18439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:56.297080 18439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:56.297175 18439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:56.297269 18439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:24:56.303057 18439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:56.304188 18439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:56.305256 18439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:56.315303 18444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:56.315284 18445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.315357 18439 server_base.cc:1034] running on GCE node
W20260619 22:24:56.315303 18447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.315954 18439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:56.316650 18439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:56.317837 18439 hybrid_clock.cc:648] HybridClock initialized: now 1781907896317806 us; error 44 us; skew 500 ppm
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:56.320920 18439 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:24:56.321900 18439 webserver.cc:460] Webserver started at http://127.1.74.129:44899/ using document root <none> and password file <none>
I20260619 22:24:56.322501 18439 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:56.322597 18439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:56.322831 18439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:56.324995 18439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "8829b2b7f0364689ba6a5dec6ddd7455"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "5a7393dacecc9cfb253491cd6a873084"
server_key_iv: "4d2429bd890bf4f3e80f5f3c8a49e272"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.325593 18439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "8829b2b7f0364689ba6a5dec6ddd7455"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "5a7393dacecc9cfb253491cd6a873084"
server_key_iv: "4d2429bd890bf4f3e80f5f3c8a49e272"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.329653 18439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 22:24:56.332314 18454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.333515 18439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:24:56.333648 18439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "8829b2b7f0364689ba6a5dec6ddd7455"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "5a7393dacecc9cfb253491cd6a873084"
server_key_iv: "4d2429bd890bf4f3e80f5f3c8a49e272"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.333766 18439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:56.358713 18439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:56.363005 18439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:56.363240 18439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:56.363878 18439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:56.364769 18439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:56.364843 18439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.364917 18439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:56.364959 18439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.374516 18439 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:40797
I20260619 22:24:56.374540 18567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:40797 every 8 connection(s)
I20260619 22:24:56.375514 18439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:24:56.382215 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18439
I20260619 22:24:56.382359 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:24:56.382678 1322 external_mini_cluster.cc:1428] Setting key 7059b9f0e4e6b6d10f1ebbe740ad1aae
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:56.390981 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.377488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:52103 (local address 127.1.74.190:37415)
0619 22:24:56.377642 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.377646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.378715 (+ 1069us) server_negotiation.cc:407] Connection header received
0619 22:24:56.380202 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.380205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.380265 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.380368 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:56.381578 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.382129 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.383750 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.384006 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.384124 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.386815 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:56.386821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:56.386827 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:56.386891 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:56.388630 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.389184 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.389189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.389192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.389293 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.389641 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.389647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.389650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.389954 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:56.390072 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.390592 (+ 520us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.390746 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:24:56.391812 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.377877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37415 (local address 127.1.74.129:52103)
0619 22:24:56.378522 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:56.378591 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:56.379971 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:56.380521 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:56.380527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:56.380893 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:56.381403 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.381418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.382291 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.382295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.383624 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.383635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.384164 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.384167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.384349 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.385001 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:56.385018 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:56.386623 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:56.388772 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:56.388778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:56.388795 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:56.389072 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:56.389406 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:56.389408 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:56.389409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:56.389544 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:56.390066 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:56.390071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:56.390334 (+ 263us) client_negotiation.cc:769] Sending connection context
0619 22:24:56.390507 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 22:24:56.390785 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":167,"threads_started":1}
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:24:56.393253 18568 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37415
I20260619 22:24:56.393529 18568 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:56.394086 18568 heartbeater.cc:511] Master 127.1.74.190:37415 requested a full tablet report, sending...
I20260619 22:24:56.395882 18377 ts_manager.cc:194] Registered new tserver with Master: 8829b2b7f0364689ba6a5dec6ddd7455 (127.1.74.129:40797)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:24:56.397544 18377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:52103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:24:56.415702 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37415
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:56.528633 18573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:56.528937 18573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:56.528982 18573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:56.533353 18573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:56.533430 18573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:56.533530 18573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:24:56.539382 18573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:56.540573 18573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:56.541621 18573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:56.551688 18581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:56.551748 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.551895 18573 server_base.cc:1034] running on GCE node
W20260619 22:24:56.551692 18578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.552475 18573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:56.553203 18573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:56.554382 18573 hybrid_clock.cc:648] HybridClock initialized: now 1781907896554355 us; error 64 us; skew 500 ppm
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:56.557341 18573 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:24:56.558317 18573 webserver.cc:460] Webserver started at http://127.1.74.130:45685/ using document root <none> and password file <none>
I20260619 22:24:56.558952 18573 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:56.559023 18573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:56.559204 18573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:56.561277 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "71f201c260c845288e249b26b339415a"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "661f55d4d7379f0682f7a644246653ec"
server_key_iv: "11462c5a7cff9afc680f3c768c842c9c"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.561803 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "71f201c260c845288e249b26b339415a"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "661f55d4d7379f0682f7a644246653ec"
server_key_iv: "11462c5a7cff9afc680f3c768c842c9c"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.565755 18573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:24:56.568549 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.569913 18573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260619 22:24:56.570115 18573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "71f201c260c845288e249b26b339415a"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "661f55d4d7379f0682f7a644246653ec"
server_key_iv: "11462c5a7cff9afc680f3c768c842c9c"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.570262 18573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:56.603960 18573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:56.607205 18573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:56.607436 18573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:56.608068 18573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:56.608958 18573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:56.609031 18573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.609123 18573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:56.609174 18573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.619233 18573 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:42415
I20260619 22:24:56.619251 18701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:42415 every 8 connection(s)
I20260619 22:24:56.620173 18573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:24:56.623080 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18573
I20260619 22:24:56.623202 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:24:56.623546 1322 external_mini_cluster.cc:1428] Setting key 4c357ffefd1db52ca8dd8c6e0e4c79c6
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:24:56.635129 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.622006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:43949 (local address 127.1.74.190:37415)
0619 22:24:56.622159 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.622162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.623093 (+ 931us) server_negotiation.cc:407] Connection header received
0619 22:24:56.624242 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.624245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.624319 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.624407 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:56.626014 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.626669 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.628023 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.628246 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.628357 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.631284 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:56.631287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:56.631289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:56.631312 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:56.632809 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.633471 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.633475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.633476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.633553 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.633937 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.633940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.633943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.634169 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:56.634279 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.634796 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.634956 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:24:56.636025 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.622365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37415 (local address 127.1.74.130:43949)
0619 22:24:56.622904 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:56.622967 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:56.624005 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:56.624567 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:56.624573 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:56.625112 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:56.625806 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.625828 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.626799 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.626803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.627901 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.627910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.628402 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.628406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.628635 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.629295 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:56.629311 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:56.631106 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:56.632965 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:56.632971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:56.632988 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:56.633345 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:56.633657 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:56.633660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:56.633662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:56.633829 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:56.634279 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:56.634287 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:56.634548 (+ 261us) client_negotiation.cc:769] Sending connection context
0619 22:24:56.634739 (+ 191us) client_negotiation.cc:240] Negotiation successful
0619 22:24:56.635101 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":106,"threads_started":1}
I20260619 22:24:56.637346 18702 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37415
I20260619 22:24:56.637622 18702 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:56.638137 18702 heartbeater.cc:511] Master 127.1.74.190:37415 requested a full tablet report, sending...
I20260619 22:24:56.639402 18377 ts_manager.cc:194] Registered new tserver with Master: 71f201c260c845288e249b26b339415a (127.1.74.130:42415)
I20260619 22:24:56.640281 18377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:43949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:24:56.654214 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37415
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:56.768168 18707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:56.768476 18707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:56.768520 18707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:56.772742 18707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:56.772827 18707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:56.772891 18707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:24:56.778702 18707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:56.779934 18707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:56.781188 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:56.792068 18713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:56.792037 18712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:56.792023 18715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:56.792702 18707 server_base.cc:1034] running on GCE node
I20260619 22:24:56.793352 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:56.794291 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:56.795593 18707 hybrid_clock.cc:648] HybridClock initialized: now 1781907896795553 us; error 57 us; skew 500 ppm
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:56.799114 18707 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:24:56.800237 18707 webserver.cc:460] Webserver started at http://127.1.74.131:36879/ using document root <none> and password file <none>
I20260619 22:24:56.800853 18707 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:56.800951 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:56.801214 18707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:56.803475 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "e8707e5243d2482d84c99bd8530b2165"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "ac428e7f2c9f839cb16585e349a1bf64"
server_key_iv: "77be1dc076f2d0f9c131a98987458830"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.804111 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "e8707e5243d2482d84c99bd8530b2165"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "ac428e7f2c9f839cb16585e349a1bf64"
server_key_iv: "77be1dc076f2d0f9c131a98987458830"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.808395 18707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:24:56.811121 18722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.812466 18707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 22:24:56.812616 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "e8707e5243d2482d84c99bd8530b2165"
format_stamp: "Formatted at 2026-06-19 22:24:56 on dist-test-slave-k84h"
server_key: "ac428e7f2c9f839cb16585e349a1bf64"
server_key_iv: "77be1dc076f2d0f9c131a98987458830"
server_key_version: "encryptionkey@0"
I20260619 22:24:56.812747 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:56.834197 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:56.837419 18707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:56.837661 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:56.838264 18707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:56.839174 18707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:56.839254 18707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.839349 18707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:56.839401 18707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:56.848624 18707 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:46657
I20260619 22:24:56.848644 18835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:46657 every 8 connection(s)
I20260619 22:24:56.849566 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:24:56.851166 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18707
I20260619 22:24:56.851310 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:24:56.851627 1322 external_mini_cluster.cc:1428] Setting key 8668a45506b5a9b69b4fafc9638b954e
Jun 19 22:24:56 dist-test-slave-k84h krb5kdc[18332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:56.863880 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.851537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:36457 (local address 127.1.74.190:37415)
0619 22:24:56.851687 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.851694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.852690 (+ 996us) server_negotiation.cc:407] Connection header received
0619 22:24:56.854163 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.854166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.854223 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.854302 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:56.855561 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.856107 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.857257 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.857484 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.857597 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.860144 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:56.860147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:56.860149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:56.860175 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:56.861802 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.862368 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.862371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.862372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.862450 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.862775 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.862778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.862779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.863029 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:56.863109 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.863624 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.863750 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:24:56.864779 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.852026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37415 (local address 127.1.74.131:36457)
0619 22:24:56.852543 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:56.852603 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:56.853963 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:56.854481 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:56.854488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:56.854927 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:56.855418 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.855432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.856235 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.856239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.857128 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.857138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.857637 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.857642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.857816 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.858395 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:24:56.858412 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:56.859959 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:56.861953 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:56.861958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:56.861972 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:56.862248 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:56.862535 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:24:56.862537 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:24:56.862538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:24:56.862671 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:24:56.863122 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:56.863127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:56.863379 (+ 252us) client_negotiation.cc:769] Sending connection context
0619 22:24:56.863548 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 22:24:56.863814 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
I20260619 22:24:56.866503 18836 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37415
I20260619 22:24:56.866911 18836 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:56.867980 18836 heartbeater.cc:511] Master 127.1.74.190:37415 requested a full tablet report, sending...
I20260619 22:24:56.869385 18376 ts_manager.cc:194] Registered new tserver with Master: e8707e5243d2482d84c99bd8530b2165 (127.1.74.131:46657)
I20260619 22:24:56.870034 18376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:36457
I20260619 22:24:56.878212 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:24:56.889962 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.880484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.1.74.190:37415)
0619 22:24:56.880708 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.880715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.880857 (+ 142us) server_negotiation.cc:407] Connection header received
0619 22:24:56.881260 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.881264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.881331 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.881424 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:56.882200 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.882736 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.883656 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.883885 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.883997 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.885689 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:56.885692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:56.885695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:56.885723 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:56.887806 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.888572 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.888576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.888578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.888689 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:56.888974 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:56.888979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:56.888980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:56.889242 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:56.889392 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.889642 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.889783 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 22:24:56.893723 18376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:24:56.897243 18376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:24:56.914721 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.908160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.1.74.130:42415)
0619 22:24:56.909306 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.909320 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.909347 (+ 27us) server_negotiation.cc:407] Connection header received
0619 22:24:56.909432 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.909438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.909670 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.909856 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:56.910185 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.911505 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.913135 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.913952 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.914189 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.914325 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.914429 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.914541 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":138,"threads_started":1}
I20260619 22:24:56.914708 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.909170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.1.74.131:46657)
0619 22:24:56.909803 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.909814 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.909861 (+ 47us) server_negotiation.cc:407] Connection header received
0619 22:24:56.909937 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.909944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.910107 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.910249 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:56.910459 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.911623 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.913095 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.913858 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.914182 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.914324 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.914429 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.914539 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":171,"threads_started":1}
I20260619 22:24:56.914727 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.908761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46657 (local address 127.0.0.1:50460)
0619 22:24:56.909310 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:56.909323 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:56.909406 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:56.910224 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:56.910228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:56.910246 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:56.910332 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.910337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.911811 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.911815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.912963 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.912973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.914054 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.914057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.914217 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.914268 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 22:24:56.914399 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 22:24:56.914584 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":53,"thread_start_us":85,"threads_started":1}
I20260619 22:24:56.914752 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.907945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42415 (local address 127.0.0.1:37372)
0619 22:24:56.908419 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:56.908477 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:56.908752 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:56.909846 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:56.909851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:56.909874 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:56.910040 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.910060 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.911658 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.911661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.913006 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.913014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.914088 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.914092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.914282 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.914305 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 22:24:56.914398 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 22:24:56.914608 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":345,"thread_start_us":116,"threads_started":1}
I20260619 22:24:56.915299 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.908351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40797 (local address 127.0.0.1:50420)
0619 22:24:56.908938 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:56.908954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:56.909047 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:56.910050 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:56.910053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:56.910068 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:56.910176 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.910181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.911295 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.911301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.912748 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:56.912760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.914008 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.914016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:56.914984 (+ 968us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.915007 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 22:24:56.915083 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 22:24:56.915164 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":33,"thread_start_us":101,"threads_started":1}
I20260619 22:24:56.915851 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:56.908832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50420 (local address 127.1.74.129:40797)
0619 22:24:56.909573 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:56.909581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:56.909605 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:24:56.909687 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:56.909692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:56.909877 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:56.910016 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:56.910303 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.911180 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.912888 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:56.913843 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:56.913964 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:56.914147 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:56.915604 (+ 1457us) server_negotiation.cc:299] Negotiation successful
0619 22:24:56.915712 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":97,"threads_started":1}
I20260619 22:24:56.918203 18636 tablet_service.cc:1467] Processing CreateTablet for tablet daa9fa02abf3414f840645bf5f76b1f2 (DEFAULT_TABLE table=test-table [id=adcf40a909cc42a292af49eedf321771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:56.918195 18770 tablet_service.cc:1467] Processing CreateTablet for tablet daa9fa02abf3414f840645bf5f76b1f2 (DEFAULT_TABLE table=test-table [id=adcf40a909cc42a292af49eedf321771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:56.918196 18502 tablet_service.cc:1467] Processing CreateTablet for tablet daa9fa02abf3414f840645bf5f76b1f2 (DEFAULT_TABLE table=test-table [id=adcf40a909cc42a292af49eedf321771]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:56.919251 18636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa9fa02abf3414f840645bf5f76b1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:56.919255 18770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa9fa02abf3414f840645bf5f76b1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:56.919255 18502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daa9fa02abf3414f840645bf5f76b1f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:56.925604 18853 tablet_bootstrap.cc:492] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Bootstrap starting.
I20260619 22:24:56.926873 18855 tablet_bootstrap.cc:492] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Bootstrap starting.
I20260619 22:24:56.927892 18853 tablet_bootstrap.cc:654] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:56.928649 18855 tablet_bootstrap.cc:654] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:56.928984 18853 log.cc:824] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:56.929683 18854 tablet_bootstrap.cc:492] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: Bootstrap starting.
I20260619 22:24:56.929778 18855 log.cc:824] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:56.931205 18853 tablet_bootstrap.cc:492] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: No bootstrap required, opened a new log
I20260619 22:24:56.931422 18853 ts_tablet_manager.cc:1397] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260619 22:24:56.931613 18855 tablet_bootstrap.cc:492] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: No bootstrap required, opened a new log
I20260619 22:24:56.931793 18855 ts_tablet_manager.cc:1397] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 22:24:56.931908 18854 tablet_bootstrap.cc:654] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:56.932564 18854 log.cc:824] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:56.934325 18854 tablet_bootstrap.cc:492] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: No bootstrap required, opened a new log
I20260619 22:24:56.934551 18854 ts_tablet_manager.cc:1397] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 22:24:56.935544 18853 raft_consensus.cc:348] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:56.935544 18855 raft_consensus.cc:348] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:56.935851 18853 raft_consensus.cc:374] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:56.935851 18855 raft_consensus.cc:374] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:56.935905 18853 raft_consensus.cc:729] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71f201c260c845288e249b26b339415a, State: Initialized, Role: FOLLOWER
I20260619 22:24:56.935912 18855 raft_consensus.cc:729] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8707e5243d2482d84c99bd8530b2165, State: Initialized, Role: FOLLOWER
I20260619 22:24:56.936437 18855 consensus_queue.cc:260] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:56.936437 18853 consensus_queue.cc:260] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:56.940501 18854 raft_consensus.cc:348] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:56.940820 18854 raft_consensus.cc:374] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:56.940862 18854 raft_consensus.cc:729] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8829b2b7f0364689ba6a5dec6ddd7455, State: Initialized, Role: FOLLOWER
I20260619 22:24:56.941385 18854 consensus_queue.cc:260] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:56.942472 18854 ts_tablet_manager.cc:1428] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260619 22:24:56.942476 18855 ts_tablet_manager.cc:1428] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260619 22:24:56.942894 18568 heartbeater.cc:503] Master 127.1.74.190:37415 was elected leader, sending a full tablet report...
I20260619 22:24:56.944887 18836 heartbeater.cc:503] Master 127.1.74.190:37415 was elected leader, sending a full tablet report...
I20260619 22:24:56.945482 18702 heartbeater.cc:503] Master 127.1.74.190:37415 was elected leader, sending a full tablet report...
I20260619 22:24:56.945786 18853 ts_tablet_manager.cc:1428] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Time spent starting tablet: real 0.014s user 0.003s sys 0.003s
I20260619 22:24:57.100346 18860 raft_consensus.cc:482] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260619 22:24:57.100556 18837 tablet.cc:2369] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:24:57.100653 18860 raft_consensus.cc:504] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:57.101934 18860 leader_election.cc:288] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8707e5243d2482d84c99bd8530b2165 (127.1.74.131:46657), 8829b2b7f0364689ba6a5dec6ddd7455 (127.1.74.129:40797)
I20260619 22:24:57.106288 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.102226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46657 (local address 127.1.74.130:32969)
0619 22:24:57.102373 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:57.102391 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:57.102498 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:57.102801 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:57.102804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:57.102822 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:57.102955 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.102960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.103704 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.103709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.104818 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.104834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.105880 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.105883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.105987 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.106009 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:24:57.106079 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 22:24:57.106146 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 22:24:57.106357 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.102304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:32969 (local address 127.1.74.131:46657)
0619 22:24:57.102439 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:57.102444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:57.102463 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:24:57.102618 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:57.102621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:57.102676 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:57.102767 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:57.103068 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.103581 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.105056 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.105748 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.105875 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.105932 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:57.106121 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 22:24:57.106223 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:24:57.106773 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.102525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:55219 (local address 127.1.74.129:40797)
0619 22:24:57.102654 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:57.102661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:57.102677 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:24:57.102883 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:57.102888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:57.102937 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:57.103025 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:57.103286 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.103793 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.105329 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.106002 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.106119 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.106165 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:57.106556 (+ 391us) server_negotiation.cc:299] Negotiation successful
0619 22:24:57.106643 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:24:57.106798 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.102229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40797 (local address 127.1.74.130:55219)
0619 22:24:57.102607 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:57.102620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:57.102726 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:57.103034 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:57.103037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:57.103052 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:57.103175 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.103180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.103912 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.103917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.105172 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.105185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.106280 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.106284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.106387 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.106412 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 22:24:57.106522 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 22:24:57.106638 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":80,"thread_start_us":89,"threads_started":1}
I20260619 22:24:57.107231 18790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa9fa02abf3414f840645bf5f76b1f2" candidate_uuid: "71f201c260c845288e249b26b339415a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8707e5243d2482d84c99bd8530b2165" is_pre_election: true
I20260619 22:24:57.107532 18790 raft_consensus.cc:2454] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71f201c260c845288e249b26b339415a in term 0.
I20260619 22:24:57.107568 18522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa9fa02abf3414f840645bf5f76b1f2" candidate_uuid: "71f201c260c845288e249b26b339415a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" is_pre_election: true
I20260619 22:24:57.107836 18522 raft_consensus.cc:2454] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71f201c260c845288e249b26b339415a in term 0.
I20260619 22:24:57.107990 18590 leader_election.cc:302] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71f201c260c845288e249b26b339415a, e8707e5243d2482d84c99bd8530b2165; no voters:
I20260619 22:24:57.108289 18860 raft_consensus.cc:2781] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:24:57.108366 18860 raft_consensus.cc:482] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:24:57.108404 18860 raft_consensus.cc:3037] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:57.109545 18860 raft_consensus.cc:504] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:57.109968 18860 leader_election.cc:288] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [CANDIDATE]: Term 1 election: Requested vote from peers e8707e5243d2482d84c99bd8530b2165 (127.1.74.131:46657), 8829b2b7f0364689ba6a5dec6ddd7455 (127.1.74.129:40797)
I20260619 22:24:57.110435 18522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa9fa02abf3414f840645bf5f76b1f2" candidate_uuid: "71f201c260c845288e249b26b339415a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8829b2b7f0364689ba6a5dec6ddd7455"
I20260619 22:24:57.110563 18790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daa9fa02abf3414f840645bf5f76b1f2" candidate_uuid: "71f201c260c845288e249b26b339415a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8707e5243d2482d84c99bd8530b2165"
I20260619 22:24:57.110601 18522 raft_consensus.cc:3037] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:57.110723 18790 raft_consensus.cc:3037] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:57.111858 18790 raft_consensus.cc:2454] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71f201c260c845288e249b26b339415a in term 1.
I20260619 22:24:57.111862 18522 raft_consensus.cc:2454] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71f201c260c845288e249b26b339415a in term 1.
I20260619 22:24:57.112301 18592 leader_election.cc:302] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71f201c260c845288e249b26b339415a, 8829b2b7f0364689ba6a5dec6ddd7455; no voters:
I20260619 22:24:57.112511 18860 raft_consensus.cc:2781] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:24:57.112774 18860 raft_consensus.cc:686] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 1 LEADER]: Becoming Leader. State: Replica: 71f201c260c845288e249b26b339415a, State: Running, Role: LEADER
I20260619 22:24:57.113190 18860 consensus_queue.cc:237] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } }
I20260619 22:24:57.117058 18375 catalog_manager.cc:5495] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a reported cstate change: term changed from 0 to 1, leader changed from <none> to 71f201c260c845288e249b26b339415a (127.1.74.130). New cstate: current_term: 1 leader_uuid: "71f201c260c845288e249b26b339415a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71f201c260c845288e249b26b339415a" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 } health_report { overall_health: UNKNOWN } } }
W20260619 22:24:57.121347 18703 tablet.cc:2369] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:24:57.126994 18569 tablet.cc:2369] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:24:57.153472 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.149661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.1.74.130:42415)
0619 22:24:57.149799 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:57.149804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:57.149819 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:24:57.149982 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:57.149984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:57.150042 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:57.150164 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:24:57.150431 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.151003 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.151901 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.152120 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.152249 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.152572 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:24:57.153027 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:24:57.153162 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:57.153231 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 22:24:57.153292 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 22:24:57.160063 18522 raft_consensus.cc:1261] T daa9fa02abf3414f840645bf5f76b1f2 P 8829b2b7f0364689ba6a5dec6ddd7455 [term 1 FOLLOWER]: Refusing update from remote peer 71f201c260c845288e249b26b339415a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:24:57.160063 18790 raft_consensus.cc:1261] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 1 FOLLOWER]: Refusing update from remote peer 71f201c260c845288e249b26b339415a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:24:57.160840 18860 consensus_queue.cc:1035] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8707e5243d2482d84c99bd8530b2165" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:24:57.161093 18863 consensus_queue.cc:1035] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8829b2b7f0364689ba6a5dec6ddd7455" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:24:57.173961 18867 mvcc.cc:205] Tried to move back new op lower bound from 7298694746758279168 to 7298694746578710528. Current Snapshot: MvccSnapshot[applied={T|T < 7298694746758279168}]
I20260619 22:24:57.181135 18376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:24:57.181380 18376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:24:57.184199 18376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4aea3038eb044837a868a352df6b3975: Sending DeleteTablet for 3 replicas of tablet daa9fa02abf3414f840645bf5f76b1f2
I20260619 22:24:57.185331 18770 tablet_service.cc:1514] Processing DeleteTablet for tablet daa9fa02abf3414f840645bf5f76b1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:50460
I20260619 22:24:57.185457 18636 tablet_service.cc:1514] Processing DeleteTablet for tablet daa9fa02abf3414f840645bf5f76b1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:37372
I20260619 22:24:57.186004 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18439
I20260619 22:24:57.185930 18502 tablet_service.cc:1514] Processing DeleteTablet for tablet daa9fa02abf3414f840645bf5f76b1f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:57 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:50420
I20260619 22:24:57.187824 18876 tablet_replica.cc:330] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: stopping tablet replica
I20260619 22:24:57.188220 18876 raft_consensus.cc:2229] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:24:57.188606 18876 raft_consensus.cc:2258] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:57.189023 18877 tablet_replica.cc:330] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: stopping tablet replica
I20260619 22:24:57.189339 18877 raft_consensus.cc:2229] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:24:57.189783 18877 raft_consensus.cc:2258] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:57.190660 18876 ts_tablet_manager.cc:1905] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:24:57.191284 18877 ts_tablet_manager.cc:1905] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 22:24:57.194795 18364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:40797 (error 108)
I20260619 22:24:57.195317 18877 ts_tablet_manager.cc:1918] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:24:57.195442 18877 log.cc:1196] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/daa9fa02abf3414f840645bf5f76b1f2
I20260619 22:24:57.195590 18876 ts_tablet_manager.cc:1918] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:24:57.195785 18876 log.cc:1196] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/daa9fa02abf3414f840645bf5f76b1f2
I20260619 22:24:57.195828 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18573
I20260619 22:24:57.195840 18877 ts_tablet_manager.cc:1939] T daa9fa02abf3414f840645bf5f76b1f2 P 71f201c260c845288e249b26b339415a: Deleting consensus metadata
I20260619 22:24:57.197312 18876 ts_tablet_manager.cc:1939] T daa9fa02abf3414f840645bf5f76b1f2 P e8707e5243d2482d84c99bd8530b2165: Deleting consensus metadata
I20260619 22:24:57.199236 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.198939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:40797 (local address 127.0.0.1:50436)
0619 22:24:57.199063 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:57.199152 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:40797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 22:24:57.199642 18364 catalog_manager.cc:4584] TS 8829b2b7f0364689ba6a5dec6ddd7455 (127.1.74.129:40797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet daa9fa02abf3414f840645bf5f76b1f2: Network error: Client connection negotiation failed: client connection to 127.1.74.129:40797: connect: Connection refused (error 111)
I20260619 22:24:57.199216 18362 catalog_manager.cc:4841] TS e8707e5243d2482d84c99bd8530b2165 (127.1.74.131:46657): tablet daa9fa02abf3414f840645bf5f76b1f2 (table test-table [id=adcf40a909cc42a292af49eedf321771]) successfully deleted
I20260619 22:24:57.202755 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18707
I20260619 22:24:57.203510 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.203070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42415 (local address 127.0.0.1:37400)
0619 22:24:57.203328 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:57.203402 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:42415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260619 22:24:57.208554 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18345
2026-06-19T22:24:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2585 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T22:24:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:24:57Z Disabled control of system clock
I20260619 22:24:57.242501 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37261
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 22:24:57.360339 18882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:57.360674 18882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:57.360713 18882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:57.365196 18882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:24:57.365283 18882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:57.365298 18882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:57.365316 18882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:24:57.365332 18882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:24:57.370729 18882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:57.372010 18882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:57.373508 18882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:57.382476 18890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:57.382499 18882 server_base.cc:1034] running on GCE node
W20260619 22:24:57.382428 18887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:57.382642 18888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:57.383477 18882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:57.384621 18882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:57.385810 18882 hybrid_clock.cc:648] HybridClock initialized: now 1781907897385799 us; error 29 us; skew 500 ppm
I20260619 22:24:57.388118 18882 webserver.cc:460] Webserver started at http://127.1.74.190:34049/ using document root <none> and password file <none>
I20260619 22:24:57.388768 18882 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:57.388844 18882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:57.389151 18882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:57.391532 18882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "143dc39298ab4d33a84befa225b34bbe"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "f9245460acb6c1b867c81b1924d775f3"
server_key_iv: "f548fb18ffbab2b52d48ceedc50e6d6b"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.392169 18882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "143dc39298ab4d33a84befa225b34bbe"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "f9245460acb6c1b867c81b1924d775f3"
server_key_iv: "f548fb18ffbab2b52d48ceedc50e6d6b"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.396390 18882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:24:57.399125 18896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.400486 18882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:24:57.400636 18882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "143dc39298ab4d33a84befa225b34bbe"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "f9245460acb6c1b867c81b1924d775f3"
server_key_iv: "f548fb18ffbab2b52d48ceedc50e6d6b"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.400764 18882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:57.413280 18882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:57.414057 18882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:57.414410 18882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:57.421993 18882 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:37261
I20260619 22:24:57.422001 18948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:37261 every 8 connection(s)
I20260619 22:24:57.423024 18882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:24:57.426267 18949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:57.429946 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18882
I20260619 22:24:57.430091 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:24:57.430413 1322 external_mini_cluster.cc:1428] Setting key d30e7e4a869ceb924de231330efd5fd9
I20260619 22:24:57.432679 18949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe: Bootstrap starting.
I20260619 22:24:57.434948 18949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:57.435683 18949 log.cc:824] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:57.437796 18949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe: No bootstrap required, opened a new log
I20260619 22:24:57.440793 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.432137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.1.74.190:37261)
0619 22:24:57.433002 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:57.433019 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:57.433056 (+ 37us) server_negotiation.cc:407] Connection header received
0619 22:24:57.433714 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:57.433731 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:57.434109 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:57.434463 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:57.434965 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.436104 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.436981 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.437653 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.437829 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.438701 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:57.438707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:57.438731 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:24:57.439240 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:57.439309 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:24:57.439336 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:57.439434 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:57.439606 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 22:24:57.439877 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":159,"threads_started":1}
I20260619 22:24:57.442389 18949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143dc39298ab4d33a84befa225b34bbe" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37261 } }
I20260619 22:24:57.442597 18949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:57.442663 18949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143dc39298ab4d33a84befa225b34bbe, State: Initialized, Role: FOLLOWER
I20260619 22:24:57.443157 18949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143dc39298ab4d33a84befa225b34bbe" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37261 } }
I20260619 22:24:57.443321 18949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:24:57.443406 18949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:24:57.443503 18949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:57.444435 18949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143dc39298ab4d33a84befa225b34bbe" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37261 } }
I20260619 22:24:57.444775 18949 leader_election.cc:302] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 143dc39298ab4d33a84befa225b34bbe; no voters:
I20260619 22:24:57.445086 18949 leader_election.cc:288] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:24:57.445245 18954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:24:57.445487 18954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [term 1 LEADER]: Becoming Leader. State: Replica: 143dc39298ab4d33a84befa225b34bbe, State: Running, Role: LEADER
I20260619 22:24:57.445804 18954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143dc39298ab4d33a84befa225b34bbe" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37261 } }
I20260619 22:24:57.446439 18949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:24:57.447698 18956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 143dc39298ab4d33a84befa225b34bbe. Latest consensus state: current_term: 1 leader_uuid: "143dc39298ab4d33a84befa225b34bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143dc39298ab4d33a84befa225b34bbe" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37261 } } }
I20260619 22:24:57.447719 18955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "143dc39298ab4d33a84befa225b34bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "143dc39298ab4d33a84befa225b34bbe" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37261 } } }
I20260619 22:24:57.447842 18956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:57.447842 18955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:57.448192 18963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:24:57.451300 18963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:24:57.456888 18963 catalog_manager.cc:1347] Generated new cluster ID: 2a63a9c59e5d49718ac470b926c4e037
I20260619 22:24:57.456974 18963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:24:57.478711 18963 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:24:57.480095 18963 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:24:57.486263 18963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe: Generated new TSK 0
I20260619 22:24:57.487162 18963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:24:57.491524 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37261
--builtin_ntp_servers=127.1.74.148:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 22:24:57.607203 18973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:57.607494 18973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:57.607535 18973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:57.611676 18973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:57.611737 18973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:57.611797 18973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:24:57.616699 18973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:57.617904 18973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:57.619380 18973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:24:57.629032 18973 server_base.cc:1034] running on GCE node
W20260619 22:24:57.628988 18978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:57.628979 18981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:57.628973 18979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:57.630155 18973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:57.631073 18973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:57.632233 18973 hybrid_clock.cc:648] HybridClock initialized: now 1781907897632205 us; error 54 us; skew 500 ppm
I20260619 22:24:57.634502 18973 webserver.cc:460] Webserver started at http://127.1.74.129:41935/ using document root <none> and password file <none>
I20260619 22:24:57.635306 18973 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:57.635409 18973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:57.635661 18973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:57.638468 18973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "5f950d462cbd44739802b1ef5b4037c0"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "d3144d247edf3af5323cb76b2ed4f5a1"
server_key_iv: "12072978d90d583cb617aa03c3cf9e82"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.639060 18973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "5f950d462cbd44739802b1ef5b4037c0"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "d3144d247edf3af5323cb76b2ed4f5a1"
server_key_iv: "12072978d90d583cb617aa03c3cf9e82"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.642668 18973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:24:57.645224 18987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.646623 18973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:24:57.646759 18973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "5f950d462cbd44739802b1ef5b4037c0"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "d3144d247edf3af5323cb76b2ed4f5a1"
server_key_iv: "12072978d90d583cb617aa03c3cf9e82"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.646955 18973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:57.663199 18973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:57.663938 18973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:57.664140 18973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:57.664736 18973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:57.665705 18973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:57.665751 18973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.665827 18973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:57.665860 18973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.674890 18973 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:38777
I20260619 22:24:57.674938 19100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:38777 every 8 connection(s)
I20260619 22:24:57.675843 18973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:24:57.678251 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 18973
I20260619 22:24:57.678412 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:24:57.678776 1322 external_mini_cluster.cc:1428] Setting key f93e670e54f510df18169d4104fedf8b
I20260619 22:24:57.681046 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37261
--builtin_ntp_servers=127.1.74.148:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 22:24:57.687341 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.677861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:51123 (local address 127.1.74.190:37261)
0619 22:24:57.678038 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:57.678045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:57.679337 (+ 1292us) server_negotiation.cc:407] Connection header received
0619 22:24:57.681128 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:57.681134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:57.681201 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:57.681296 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:57.682608 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.683561 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.684814 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.685248 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.685384 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.686646 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:57.686653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:57.686659 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:24:57.686681 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:57.686710 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:24:57.686723 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:57.686825 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:57.687025 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 22:24:57.687111 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 22:24:57.688161 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.678255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37261 (local address 127.1.74.129:51123)
0619 22:24:57.679124 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:57.679186 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:57.680852 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:57.681522 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:57.681534 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:57.681658 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:57.682419 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.682444 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.683717 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.683720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.684664 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.684674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.685425 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.685428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.685589 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.686345 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:24:57.686346 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:57.686393 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:24:57.686468 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:57.686819 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:57.686825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:57.686829 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 22:24:57.686995 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 22:24:57.687255 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":146,"threads_started":1}
I20260619 22:24:57.689540 19101 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37261
I20260619 22:24:57.689785 19101 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:57.690356 19101 heartbeater.cc:511] Master 127.1.74.190:37261 requested a full tablet report, sending...
I20260619 22:24:57.691939 18913 ts_manager.cc:194] Registered new tserver with Master: 5f950d462cbd44739802b1ef5b4037c0 (127.1.74.129:38777)
I20260619 22:24:57.693121 18913 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.129:51123
W20260619 22:24:57.799640 19104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:57.800103 19104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:57.800194 19104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:57.804659 19104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:57.804721 19104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:57.804819 19104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:24:57.810022 19104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:57.811296 19104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:57.812754 19104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:57.823136 19110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:57.823146 19112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:57.823136 19109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:57.823634 19104 server_base.cc:1034] running on GCE node
I20260619 22:24:57.824139 19104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:57.824942 19104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:57.826203 19104 hybrid_clock.cc:648] HybridClock initialized: now 1781907897826179 us; error 39 us; skew 500 ppm
I20260619 22:24:57.828478 19104 webserver.cc:460] Webserver started at http://127.1.74.130:46253/ using document root <none> and password file <none>
I20260619 22:24:57.829109 19104 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:57.829176 19104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:57.829427 19104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:57.831861 19104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "18e299565b274bd9b6c5dcc2291c4164"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "6e57fd6f04cc7569baa745ddf6b19ee3"
server_key_iv: "4380057fa7b9f66fae956aba9f7dc81d"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.832502 19104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "18e299565b274bd9b6c5dcc2291c4164"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "6e57fd6f04cc7569baa745ddf6b19ee3"
server_key_iv: "4380057fa7b9f66fae956aba9f7dc81d"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.837270 19104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:24:57.840437 19118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.841965 19104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:24:57.842142 19104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "18e299565b274bd9b6c5dcc2291c4164"
format_stamp: "Formatted at 2026-06-19 22:24:57 on dist-test-slave-k84h"
server_key: "6e57fd6f04cc7569baa745ddf6b19ee3"
server_key_iv: "4380057fa7b9f66fae956aba9f7dc81d"
server_key_version: "encryptionkey@0"
I20260619 22:24:57.842285 19104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:57.874059 19104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:57.874962 19104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:57.875182 19104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:57.875867 19104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:57.876878 19104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:57.876951 19104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.877022 19104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:57.877069 19104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:57.888525 19104 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:42271
I20260619 22:24:57.888689 19231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:42271 every 8 connection(s)
I20260619 22:24:57.889863 19104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:24:57.899156 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19104
I20260619 22:24:57.899317 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:24:57.899658 1322 external_mini_cluster.cc:1428] Setting key 447dd7452ee65f43908d6ff7dc9bb4c9
I20260619 22:24:57.902196 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37261
--builtin_ntp_servers=127.1.74.148:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 22:24:57.903604 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.892130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:33115 (local address 127.1.74.190:37261)
0619 22:24:57.892305 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:57.892310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:57.893835 (+ 1525us) server_negotiation.cc:407] Connection header received
0619 22:24:57.896154 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:57.896158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:57.896210 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:57.896294 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:57.897764 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.898315 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.900376 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.900639 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.900763 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.903026 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:57.903030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:57.903032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:24:57.903047 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:57.903070 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:24:57.903079 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:57.903165 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:57.903373 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 22:24:57.903462 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 22:24:57.904708 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:57.892610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37261 (local address 127.1.74.130:33115)
0619 22:24:57.893570 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:57.893647 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:57.895825 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:57.896557 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:57.896571 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:57.896707 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:57.897548 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.897569 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.898457 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.898464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.900232 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:57.900247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:57.900813 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:57.900818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:57.901001 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:57.902314 (+ 1313us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:24:57.902317 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:57.902426 (+ 109us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:24:57.902586 (+ 160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:57.903186 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:57.903200 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:57.903213 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 22:24:57.903363 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 22:24:57.903726 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":145,"threads_started":1}
I20260619 22:24:57.906070 19232 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37261
I20260619 22:24:57.906344 19232 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:57.906989 19232 heartbeater.cc:511] Master 127.1.74.190:37261 requested a full tablet report, sending...
I20260619 22:24:57.908149 18913 ts_manager.cc:194] Registered new tserver with Master: 18e299565b274bd9b6c5dcc2291c4164 (127.1.74.130:42271)
I20260619 22:24:57.908759 18913 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.130:33115
W20260619 22:24:58.052590 19235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:58.052896 19235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:58.052937 19235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:58.057289 19235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:58.057376 19235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:58.057438 19235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:24:58.062602 19235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:37261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:58.063840 19235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:58.065274 19235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:58.075253 19240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:58.075366 19235 server_base.cc:1034] running on GCE node
W20260619 22:24:58.075232 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:58.075232 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:58.076061 19235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:58.076778 19235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:58.077940 19235 hybrid_clock.cc:648] HybridClock initialized: now 1781907898077923 us; error 30 us; skew 500 ppm
I20260619 22:24:58.080220 19235 webserver.cc:460] Webserver started at http://127.1.74.131:39473/ using document root <none> and password file <none>
I20260619 22:24:58.080878 19235 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:58.080946 19235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:58.081169 19235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:58.083495 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "6bb8427268fb48848c5f7d2eaaecdb8f"
format_stamp: "Formatted at 2026-06-19 22:24:58 on dist-test-slave-k84h"
server_key: "ac85417197b5eb0961075edf2b1780e8"
server_key_iv: "0eebb8cc1601b06ce732d629182f6b70"
server_key_version: "encryptionkey@0"
I20260619 22:24:58.084154 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "6bb8427268fb48848c5f7d2eaaecdb8f"
format_stamp: "Formatted at 2026-06-19 22:24:58 on dist-test-slave-k84h"
server_key: "ac85417197b5eb0961075edf2b1780e8"
server_key_iv: "0eebb8cc1601b06ce732d629182f6b70"
server_key_version: "encryptionkey@0"
I20260619 22:24:58.088196 19235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:24:58.090921 19249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:58.092278 19235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:24:58.092429 19235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "6bb8427268fb48848c5f7d2eaaecdb8f"
format_stamp: "Formatted at 2026-06-19 22:24:58 on dist-test-slave-k84h"
server_key: "ac85417197b5eb0961075edf2b1780e8"
server_key_iv: "0eebb8cc1601b06ce732d629182f6b70"
server_key_version: "encryptionkey@0"
I20260619 22:24:58.092563 19235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:58.112792 19235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:58.113550 19235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:58.113757 19235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:58.114365 19235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:24:58.115382 19235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:24:58.115455 19235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:58.115528 19235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:24:58.115572 19235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:58.125087 19235 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:45413
I20260619 22:24:58.125111 19362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:45413 every 8 connection(s)
I20260619 22:24:58.126114 19235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:24:58.131489 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19235
I20260619 22:24:58.131610 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:24:58.131888 1322 external_mini_cluster.cc:1428] Setting key 86af6b5bbd9fc1234b2d74f5013daac2
I20260619 22:24:58.136301 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.128187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:44855 (local address 127.1.74.190:37261)
0619 22:24:58.128377 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.128382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.129519 (+ 1137us) server_negotiation.cc:407] Connection header received
0619 22:24:58.130956 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.130958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.131013 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.131090 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:58.132196 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.132764 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.134085 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.134411 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.134474 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.135777 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:58.135781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:58.135785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:24:58.135804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:58.135822 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:24:58.135829 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:58.135911 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.136085 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.136152 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 22:24:58.137265 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.128640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37261 (local address 127.1.74.131:44855)
0619 22:24:58.129339 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.129384 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:58.130694 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:58.131239 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:58.131246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:58.131339 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:24:58.132019 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.132046 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.132925 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.132933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.133925 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.133940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.134643 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.134645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.134802 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.135488 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:24:58.135490 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:24:58.135534 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:24:58.135610 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:24:58.135926 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:24:58.135937 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:24:58.135949 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 22:24:58.136083 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 22:24:58.136398 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":133,"threads_started":1}
I20260619 22:24:58.138433 19363 heartbeater.cc:347] Connected to a master server at 127.1.74.190:37261
I20260619 22:24:58.138700 19363 heartbeater.cc:464] Registering TS with master...
I20260619 22:24:58.139354 19363 heartbeater.cc:511] Master 127.1.74.190:37261 requested a full tablet report, sending...
I20260619 22:24:58.140451 18913 ts_manager.cc:194] Registered new tserver with Master: 6bb8427268fb48848c5f7d2eaaecdb8f (127.1.74.131:45413)
I20260619 22:24:58.141047 18913 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.131:44855
I20260619 22:24:58.146382 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:24:58.152985 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.148695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.1.74.190:37261)
0619 22:24:58.148885 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.148889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.149035 (+ 146us) server_negotiation.cc:407] Connection header received
0619 22:24:58.149206 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.149210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.149263 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.149358 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:58.149676 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.150282 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.151309 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.151549 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.151659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.152536 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:58.152539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:58.152542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:24:58.152557 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:58.152586 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:24:58.152594 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:58.152673 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.152765 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.152826 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:24:58.156090 18913 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:24:58.159433 18913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:24:58.175688 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.169254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:38777 (local address 127.0.0.1:57710)
0619 22:24:58.170524 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.170585 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:58.170726 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:58.171706 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:58.171709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:58.171733 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:58.171937 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.171942 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.173054 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.173058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.174134 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.174144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.175023 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.175027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.175162 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.175203 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 22:24:58.175332 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 22:24:58.175476 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":70,"spinlock_wait_cycles":7040,"thread_start_us":106,"threads_started":1}
I20260619 22:24:58.175678 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.169387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.1.74.129:38777)
0619 22:24:58.171051 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.171060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.171086 (+ 26us) server_negotiation.cc:407] Connection header received
0619 22:24:58.171166 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.171171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.171392 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.171552 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:58.172080 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.172935 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.174266 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.174833 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.175024 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.175256 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.175361 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.175474 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1551,"thread_start_us":128,"threads_started":1}
I20260619 22:24:58.175740 19374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.169384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42271 (local address 127.0.0.1:34476)
0619 22:24:58.170538 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.170585 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:58.170726 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:58.171572 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:58.171590 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:58.171616 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:58.171778 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.171785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.173308 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.173311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.174163 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.174173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.175274 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.175278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.175393 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.175421 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 22:24:58.175482 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 22:24:58.175558 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":38,"spinlock_wait_cycles":3840,"thread_start_us":99,"threads_started":1}
I20260619 22:24:58.176601 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.170133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.1.74.130:42271)
0619 22:24:58.171050 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.171061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.171085 (+ 24us) server_negotiation.cc:407] Connection header received
0619 22:24:58.171166 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.171171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.171402 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.171552 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:58.171901 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.173162 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.174299 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.175133 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.176076 (+ 943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.176276 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.176327 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.176425 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":132,"threads_started":1}
I20260619 22:24:58.177498 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.170582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:45413 (local address 127.0.0.1:40366)
0619 22:24:58.170929 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.170944 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:58.171033 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:58.171711 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:58.171713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:58.171739 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:58.171934 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.171941 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.173305 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.173308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.175390 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.175398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.177064 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.177068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.177158 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.177182 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:24:58.177257 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 22:24:58.177340 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":65,"spinlock_wait_cycles":9344,"thread_start_us":106,"threads_started":1}
I20260619 22:24:58.178012 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.170801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.1.74.131:45413)
0619 22:24:58.171229 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.171238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.171263 (+ 25us) server_negotiation.cc:407] Connection header received
0619 22:24:58.171322 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.171325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.171460 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.171626 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:58.172058 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.173183 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.175524 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.176570 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.176685 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.177002 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.177713 (+ 711us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.177845 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":119,"threads_started":1}
I20260619 22:24:58.179405 19166 tablet_service.cc:1467] Processing CreateTablet for tablet db97c94d0798497f838ca56cc6c9b1c4 (DEFAULT_TABLE table=test-table [id=57df9425153646c7a24964fa27a9b7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:58.179397 19035 tablet_service.cc:1467] Processing CreateTablet for tablet db97c94d0798497f838ca56cc6c9b1c4 (DEFAULT_TABLE table=test-table [id=57df9425153646c7a24964fa27a9b7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:58.180684 19166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db97c94d0798497f838ca56cc6c9b1c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:58.180684 19035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db97c94d0798497f838ca56cc6c9b1c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:58.180546 19297 tablet_service.cc:1467] Processing CreateTablet for tablet db97c94d0798497f838ca56cc6c9b1c4 (DEFAULT_TABLE table=test-table [id=57df9425153646c7a24964fa27a9b7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:24:58.181484 19297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db97c94d0798497f838ca56cc6c9b1c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:58.189061 19380 tablet_bootstrap.cc:492] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Bootstrap starting.
I20260619 22:24:58.189424 19381 tablet_bootstrap.cc:492] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0: Bootstrap starting.
I20260619 22:24:58.191241 19382 tablet_bootstrap.cc:492] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Bootstrap starting.
I20260619 22:24:58.191313 19380 tablet_bootstrap.cc:654] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:58.191345 19381 tablet_bootstrap.cc:654] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:58.192119 19380 log.cc:824] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:58.192165 19381 log.cc:824] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:58.193459 19382 tablet_bootstrap.cc:654] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:58.194320 19382 log.cc:824] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:58.194635 19380 tablet_bootstrap.cc:492] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: No bootstrap required, opened a new log
I20260619 22:24:58.194898 19380 ts_tablet_manager.cc:1397] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:24:58.195402 19381 tablet_bootstrap.cc:492] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0: No bootstrap required, opened a new log
I20260619 22:24:58.195598 19381 ts_tablet_manager.cc:1397] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:24:58.196636 19382 tablet_bootstrap.cc:492] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: No bootstrap required, opened a new log
I20260619 22:24:58.196867 19382 ts_tablet_manager.cc:1397] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:24:58.199064 19380 raft_consensus.cc:348] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.199385 19380 raft_consensus.cc:374] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:58.199441 19380 raft_consensus.cc:729] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb8427268fb48848c5f7d2eaaecdb8f, State: Initialized, Role: FOLLOWER
I20260619 22:24:58.199915 19380 consensus_queue.cc:260] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.200067 19381 raft_consensus.cc:348] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.200331 19381 raft_consensus.cc:374] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:58.200392 19381 raft_consensus.cc:729] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f950d462cbd44739802b1ef5b4037c0, State: Initialized, Role: FOLLOWER
I20260619 22:24:58.200757 19363 heartbeater.cc:503] Master 127.1.74.190:37261 was elected leader, sending a full tablet report...
I20260619 22:24:58.200865 19381 consensus_queue.cc:260] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.201030 19380 ts_tablet_manager.cc:1428] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:24:58.201327 19382 raft_consensus.cc:348] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.201577 19382 raft_consensus.cc:374] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:58.201623 19382 raft_consensus.cc:729] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e299565b274bd9b6c5dcc2291c4164, State: Initialized, Role: FOLLOWER
I20260619 22:24:58.201856 19381 ts_tablet_manager.cc:1428] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260619 22:24:58.202028 19382 consensus_queue.cc:260] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.202196 19101 heartbeater.cc:503] Master 127.1.74.190:37261 was elected leader, sending a full tablet report...
I20260619 22:24:58.202680 19232 heartbeater.cc:503] Master 127.1.74.190:37261 was elected leader, sending a full tablet report...
I20260619 22:24:58.203007 19382 ts_tablet_manager.cc:1428] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 22:24:58.278220 19386 raft_consensus.cc:482] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:24:58.278450 19386 raft_consensus.cc:504] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.279620 19386 leader_election.cc:288] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f950d462cbd44739802b1ef5b4037c0 (127.1.74.129:38777), 18e299565b274bd9b6c5dcc2291c4164 (127.1.74.130:42271)
I20260619 22:24:58.283782 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.279903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:38777 (local address 127.1.74.131:48413)
0619 22:24:58.280146 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.280162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:58.280304 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:58.280596 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:58.280599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:58.280619 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:58.280765 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.280773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.281623 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.281627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.282513 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.282523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.283313 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.283317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.283428 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.283465 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 22:24:58.283547 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 22:24:58.283633 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260619 22:24:58.283810 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.279971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:48413 (local address 127.1.74.129:38777)
0619 22:24:58.280100 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.280105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.280270 (+ 165us) server_negotiation.cc:407] Connection header received
0619 22:24:58.280425 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.280431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.280487 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.280566 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:58.280902 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.281503 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.282647 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.283192 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.283288 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.283329 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.283576 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.283648 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:24:58.283972 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.279903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:42271 (local address 127.1.74.131:53725)
0619 22:24:58.280326 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.280361 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 22:24:58.280486 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:24:58.280846 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:24:58.280852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:24:58.280877 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:24:58.281000 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.281007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.281875 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.281877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.282688 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:24:58.282698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.283564 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.283568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:24:58.283664 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.283685 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 22:24:58.283762 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 22:24:58.283825 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":50,"thread_start_us":139,"threads_started":1}
I20260619 22:24:58.284009 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.279971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:53725 (local address 127.1.74.130:42271)
0619 22:24:58.280100 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.280105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.280508 (+ 403us) server_negotiation.cc:407] Connection header received
0619 22:24:58.280604 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.280610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.280658 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.280748 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:24:58.281149 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.281757 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.282933 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.283416 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.283556 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.283598 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.283795 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.283882 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 22:24:58.284530 19055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db97c94d0798497f838ca56cc6c9b1c4" candidate_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f950d462cbd44739802b1ef5b4037c0" is_pre_election: true
I20260619 22:24:58.284658 19186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db97c94d0798497f838ca56cc6c9b1c4" candidate_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e299565b274bd9b6c5dcc2291c4164" is_pre_election: true
I20260619 22:24:58.284883 19055 raft_consensus.cc:2454] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb8427268fb48848c5f7d2eaaecdb8f in term 0.
I20260619 22:24:58.284971 19186 raft_consensus.cc:2454] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bb8427268fb48848c5f7d2eaaecdb8f in term 0.
I20260619 22:24:58.285455 19253 leader_election.cc:302] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18e299565b274bd9b6c5dcc2291c4164, 5f950d462cbd44739802b1ef5b4037c0, 6bb8427268fb48848c5f7d2eaaecdb8f; no voters:
I20260619 22:24:58.285765 19386 raft_consensus.cc:2781] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:24:58.285863 19386 raft_consensus.cc:482] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:24:58.285925 19386 raft_consensus.cc:3037] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:58.286967 19386 raft_consensus.cc:504] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.287328 19386 leader_election.cc:288] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [CANDIDATE]: Term 1 election: Requested vote from peers 5f950d462cbd44739802b1ef5b4037c0 (127.1.74.129:38777), 18e299565b274bd9b6c5dcc2291c4164 (127.1.74.130:42271)
I20260619 22:24:58.287737 19186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db97c94d0798497f838ca56cc6c9b1c4" candidate_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18e299565b274bd9b6c5dcc2291c4164"
I20260619 22:24:58.287798 19055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db97c94d0798497f838ca56cc6c9b1c4" candidate_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f950d462cbd44739802b1ef5b4037c0"
I20260619 22:24:58.287855 19186 raft_consensus.cc:3037] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:58.287940 19055 raft_consensus.cc:3037] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:58.288921 19055 raft_consensus.cc:2454] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb8427268fb48848c5f7d2eaaecdb8f in term 1.
I20260619 22:24:58.288928 19186 raft_consensus.cc:2454] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bb8427268fb48848c5f7d2eaaecdb8f in term 1.
I20260619 22:24:58.289248 19253 leader_election.cc:302] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f950d462cbd44739802b1ef5b4037c0, 6bb8427268fb48848c5f7d2eaaecdb8f; no voters:
I20260619 22:24:58.289427 19386 raft_consensus.cc:2781] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:24:58.289657 19386 raft_consensus.cc:686] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 1 LEADER]: Becoming Leader. State: Replica: 6bb8427268fb48848c5f7d2eaaecdb8f, State: Running, Role: LEADER
I20260619 22:24:58.289983 19386 consensus_queue.cc:237] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } }
I20260619 22:24:58.293229 18913 catalog_manager.cc:5495] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bb8427268fb48848c5f7d2eaaecdb8f (127.1.74.131). New cstate: current_term: 1 leader_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bb8427268fb48848c5f7d2eaaecdb8f" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 45413 } health_report { overall_health: HEALTHY } } }
I20260619 22:24:58.306960 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.303202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.1.74.131:45413)
0619 22:24:58.303366 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:58.303370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:58.303382 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:24:58.303451 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:58.303453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:58.303503 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:58.303590 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:24:58.303879 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.304463 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.305399 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:58.305602 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:58.305681 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:58.305998 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:24:58.306447 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:24:58.306552 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:58.306673 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 22:24:58.306720 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 22:24:58.313154 19186 raft_consensus.cc:1261] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 1 FOLLOWER]: Refusing update from remote peer 6bb8427268fb48848c5f7d2eaaecdb8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:24:58.313159 19055 raft_consensus.cc:1261] T db97c94d0798497f838ca56cc6c9b1c4 P 5f950d462cbd44739802b1ef5b4037c0 [term 1 FOLLOWER]: Refusing update from remote peer 6bb8427268fb48848c5f7d2eaaecdb8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:24:58.313987 19390 consensus_queue.cc:1035] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f950d462cbd44739802b1ef5b4037c0" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 38777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:24:58.314215 19386 consensus_queue.cc:1035] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18e299565b274bd9b6c5dcc2291c4164" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 42271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:24:58.332118 18913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:24:58.332278 18913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:24:58.334718 18913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 143dc39298ab4d33a84befa225b34bbe: Sending DeleteTablet for 3 replicas of tablet db97c94d0798497f838ca56cc6c9b1c4
I20260619 22:24:58.335539 19297 tablet_service.cc:1514] Processing DeleteTablet for tablet db97c94d0798497f838ca56cc6c9b1c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:58 UTC) from {username='slave'} at 127.0.0.1:40366
I20260619 22:24:58.335558 19035 tablet_service.cc:1514] Processing DeleteTablet for tablet db97c94d0798497f838ca56cc6c9b1c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:58 UTC) from {username='slave'} at 127.0.0.1:57710
I20260619 22:24:58.335558 19166 tablet_service.cc:1514] Processing DeleteTablet for tablet db97c94d0798497f838ca56cc6c9b1c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:24:58 UTC) from {username='slave'} at 127.0.0.1:34476
I20260619 22:24:58.336089 19403 tablet_replica.cc:330] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: stopping tablet replica
I20260619 22:24:58.336263 19402 tablet_replica.cc:330] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: stopping tablet replica
I20260619 22:24:58.336529 19403 raft_consensus.cc:2229] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:24:58.336529 19402 raft_consensus.cc:2229] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:24:58.336855 19402 raft_consensus.cc:2258] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:58.336907 19403 raft_consensus.cc:2258] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:24:58.336982 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18973
I20260619 22:24:58.338826 19403 ts_tablet_manager.cc:1905] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:24:58.338826 19402 ts_tablet_manager.cc:1905] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:24:58.341564 19402 ts_tablet_manager.cc:1918] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:24:58.341564 19403 ts_tablet_manager.cc:1918] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:24:58.341670 19402 log.cc:1196] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/db97c94d0798497f838ca56cc6c9b1c4
I20260619 22:24:58.341670 19403 log.cc:1196] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/db97c94d0798497f838ca56cc6c9b1c4
I20260619 22:24:58.342010 19403 ts_tablet_manager.cc:1939] T db97c94d0798497f838ca56cc6c9b1c4 P 18e299565b274bd9b6c5dcc2291c4164: Deleting consensus metadata
I20260619 22:24:58.342010 19402 ts_tablet_manager.cc:1939] T db97c94d0798497f838ca56cc6c9b1c4 P 6bb8427268fb48848c5f7d2eaaecdb8f: Deleting consensus metadata
I20260619 22:24:58.343263 18899 catalog_manager.cc:4841] TS 6bb8427268fb48848c5f7d2eaaecdb8f (127.1.74.131:45413): tablet db97c94d0798497f838ca56cc6c9b1c4 (table test-table [id=57df9425153646c7a24964fa27a9b7cc]) successfully deleted
I20260619 22:24:58.343263 18897 catalog_manager.cc:4841] TS 18e299565b274bd9b6c5dcc2291c4164 (127.1.74.130:42271): tablet db97c94d0798497f838ca56cc6c9b1c4 (table test-table [id=57df9425153646c7a24964fa27a9b7cc]) successfully deleted
W20260619 22:24:58.348397 18900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:38777 (error 108)
I20260619 22:24:58.348588 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19104
I20260619 22:24:58.350063 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:58.349792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:38777 (local address 127.0.0.1:57714)
0619 22:24:58.349916 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 22:24:58.349987 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:38777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 22:24:58.350257 18900 catalog_manager.cc:4584] TS 5f950d462cbd44739802b1ef5b4037c0 (127.1.74.129:38777): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db97c94d0798497f838ca56cc6c9b1c4: Network error: Client connection negotiation failed: client connection to 127.1.74.129:38777: connect: Connection refused (error 111)
I20260619 22:24:58.357342 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19235
I20260619 22:24:58.362912 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 18882
2026-06-19T22:24:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:24:58 dist-test-slave-k84h krb5kdc[19408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:24:58 dist-test-slave-k84h krb5kdc[19408](info): set up 2 sockets
Jun 19 22:24:58 dist-test-slave-k84h krb5kdc[19408](info): commencing operation
krb5kdc: starting...
W20260619 22:24:59.415063 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:24:59 dist-test-slave-k84h krb5kdc[19408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:24:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:24:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:24:59.543256 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44793
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:44793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:59.658809 19421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:59.659195 19421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:59.659291 19421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:59.663496 19421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:24:59.663596 19421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:59.663635 19421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:59.663694 19421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:24:59.663731 19421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:24:59.669417 19421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:44793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:59.670584 19421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:59.671721 19421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:59.680822 19426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:59.680804 19429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:59.680976 19421 server_base.cc:1034] running on GCE node
W20260619 22:24:59.680783 19427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:59.681552 19421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:59.682790 19421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:59.684023 19421 hybrid_clock.cc:648] HybridClock initialized: now 1781907899683995 us; error 41 us; skew 500 ppm
Jun 19 22:24:59 dist-test-slave-k84h krb5kdc[19408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:59.691509 19421 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:24:59.692476 19421 webserver.cc:460] Webserver started at http://127.1.74.190:36313/ using document root <none> and password file <none>
I20260619 22:24:59.693023 19421 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:59.693094 19421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:59.693260 19421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:59.695365 19421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "1dab987d44ba48f4a6f77baf4db5cc8f"
format_stamp: "Formatted at 2026-06-19 22:24:59 on dist-test-slave-k84h"
server_key: "83e81a21a1ec7e94ceebd9e9c09a75fd"
server_key_iv: "6b4ae96bed14af0da9d454a9537ba315"
server_key_version: "encryptionkey@0"
I20260619 22:24:59.695899 19421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "1dab987d44ba48f4a6f77baf4db5cc8f"
format_stamp: "Formatted at 2026-06-19 22:24:59 on dist-test-slave-k84h"
server_key: "83e81a21a1ec7e94ceebd9e9c09a75fd"
server_key_iv: "6b4ae96bed14af0da9d454a9537ba315"
server_key_version: "encryptionkey@0"
I20260619 22:24:59.699800 19421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:24:59.702301 19436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:59.703591 19421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:24:59.703698 19421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "1dab987d44ba48f4a6f77baf4db5cc8f"
format_stamp: "Formatted at 2026-06-19 22:24:59 on dist-test-slave-k84h"
server_key: "83e81a21a1ec7e94ceebd9e9c09a75fd"
server_key_iv: "6b4ae96bed14af0da9d454a9537ba315"
server_key_version: "encryptionkey@0"
I20260619 22:24:59.703797 19421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:24:59.715839 19421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:24:59.719223 19421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:24:59.719388 19421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:24:59.726532 19421 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:44793
I20260619 22:24:59.726626 19488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:44793 every 8 connection(s)
I20260619 22:24:59.727588 19421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:24:59.729960 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19421
I20260619 22:24:59.730129 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:24:59.730487 1322 external_mini_cluster.cc:1428] Setting key a9c2300b8bc654bee4c1f3c3eab05fd7
I20260619 22:24:59.730888 19489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:24:59.737432 19489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f: Bootstrap starting.
Jun 19 22:24:59 dist-test-slave-k84h krb5kdc[19408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:24:59.739969 19489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f: Neither blocks nor log segments found. Creating new log.
I20260619 22:24:59.740686 19489 log.cc:824] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:24:59.742596 19489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f: No bootstrap required, opened a new log
I20260619 22:24:59.745007 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:24:59.732068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.1.74.190:44793)
0619 22:24:59.732601 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0619 22:24:59.732620 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:24:59.732656 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:24:59.733350 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:24:59.733366 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:24:59.733705 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:24:59.734078 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:24:59.734945 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:59.735948 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:59.736833 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:24:59.737329 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:24:59.737508 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:24:59.740182 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:24:59.740187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:24:59.740200 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:24:59.740227 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:24:59.741995 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:59.742447 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:59.742453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:59.742458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:59.742574 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:24:59.742926 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:24:59.742929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:24:59.742931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:24:59.743344 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:24:59.743550 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 22:24:59.743864 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 22:24:59.744162 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":186,"threads_started":1}
I20260619 22:24:59.746320 19489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44793 } }
I20260619 22:24:59.746572 19489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:24:59.746654 19489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dab987d44ba48f4a6f77baf4db5cc8f, State: Initialized, Role: FOLLOWER
I20260619 22:24:59.747171 19489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44793 } }
I20260619 22:24:59.747431 19489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:24:59.747542 19489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:24:59.747642 19489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:24:59.748556 19489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44793 } }
I20260619 22:24:59.748852 19489 leader_election.cc:302] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dab987d44ba48f4a6f77baf4db5cc8f; no voters:
I20260619 22:24:59.749114 19489 leader_election.cc:288] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:24:59.749246 19494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:24:59.749516 19494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [term 1 LEADER]: Becoming Leader. State: Replica: 1dab987d44ba48f4a6f77baf4db5cc8f, State: Running, Role: LEADER
I20260619 22:24:59.749919 19494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44793 } }
I20260619 22:24:59.750396 19489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:24:59.751559 19495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44793 } } }
I20260619 22:24:59.751706 19495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:59.752053 19496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dab987d44ba48f4a6f77baf4db5cc8f. Latest consensus state: current_term: 1 leader_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dab987d44ba48f4a6f77baf4db5cc8f" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44793 } } }
I20260619 22:24:59.752143 19503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:24:59.752218 19496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f [sys.catalog]: This master's current role is: LEADER
I20260619 22:24:59.755410 19503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:24:59.761046 19503 catalog_manager.cc:1347] Generated new cluster ID: 765c8162a0b247f9bf7f46f82f5fe5a6
I20260619 22:24:59.761173 19503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:24:59.785705 19503 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:24:59.786707 19503 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:24:59.792434 19503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f: Generated new TSK 0
I20260619 22:24:59.793255 19503 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:24:59.827030 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44793
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:24:59.952672 19515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:24:59.953020 19515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:24:59.953107 19515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:24:59.957459 19515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:24:59.957579 19515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:24:59.957679 19515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:24:59.963582 19515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:24:59.964785 19515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:24:59.965855 19515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:24:59.976923 19523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:59.976922 19521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:24:59.976964 19520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:24:59.976992 19515 server_base.cc:1034] running on GCE node
I20260619 22:24:59.977617 19515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:24:59.978319 19515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:24:59.979537 19515 hybrid_clock.cc:648] HybridClock initialized: now 1781907899979511 us; error 42 us; skew 500 ppm
Jun 19 22:24:59 dist-test-slave-k84h krb5kdc[19408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:24:59.982818 19515 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:24:59.983925 19515 webserver.cc:460] Webserver started at http://127.1.74.129:35989/ using document root <none> and password file <none>
I20260619 22:24:59.984613 19515 fs_manager.cc:359] Metadata directory not provided
I20260619 22:24:59.984719 19515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:24:59.984995 19515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:24:59.987349 19515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "fa803eef014e4e62a13c313e75ad0958"
format_stamp: "Formatted at 2026-06-19 22:24:59 on dist-test-slave-k84h"
server_key: "b2acce97124f5d5552e455dfb4574047"
server_key_iv: "bda0d38d9fecb925aa963bc23f3d46f7"
server_key_version: "encryptionkey@0"
I20260619 22:24:59.988006 19515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "fa803eef014e4e62a13c313e75ad0958"
format_stamp: "Formatted at 2026-06-19 22:24:59 on dist-test-slave-k84h"
server_key: "b2acce97124f5d5552e455dfb4574047"
server_key_iv: "bda0d38d9fecb925aa963bc23f3d46f7"
server_key_version: "encryptionkey@0"
I20260619 22:24:59.992247 19515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 22:24:59.995428 19530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:24:59.996938 19515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:24:59.997105 19515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "fa803eef014e4e62a13c313e75ad0958"
format_stamp: "Formatted at 2026-06-19 22:24:59 on dist-test-slave-k84h"
server_key: "b2acce97124f5d5552e455dfb4574047"
server_key_iv: "bda0d38d9fecb925aa963bc23f3d46f7"
server_key_version: "encryptionkey@0"
I20260619 22:24:59.997257 19515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:00.038663 19515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:00.042073 19515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:00.042303 19515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:00.042936 19515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:00.043776 19515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:00.043854 19515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.043931 19515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:00.043982 19515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.053540 19515 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:43023
I20260619 22:25:00.053567 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:43023 every 8 connection(s)
I20260619 22:25:00.054445 19515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:25:00.055524 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19515
I20260619 22:25:00.055668 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:25:00.056010 1322 external_mini_cluster.cc:1428] Setting key 9886e4bd3865777f78ce7ff59e7d6a6d
Jun 19 22:25:00 dist-test-slave-k84h krb5kdc[19408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:25:00.069265 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.056485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:42501 (local address 127.1.74.190:44793)
0619 22:25:00.056626 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.056630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.057477 (+ 847us) server_negotiation.cc:407] Connection header received
0619 22:25:00.058835 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.058840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.058927 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.058987 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:00.060642 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.061183 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.062320 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.062561 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.062677 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.065172 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:00.065175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:00.065177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:00.065199 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:00.066812 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.067492 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.067495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.067496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.067571 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.068075 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.068078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.068080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.068278 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:00.068371 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.069007 (+ 636us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.069127 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:25:00.070327 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.056870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44793 (local address 127.1.74.129:42501)
0619 22:25:00.057320 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.057388 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.058595 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.059426 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.059436 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.059960 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:00.060499 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.060512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.061336 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.061340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.062194 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.062203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.062717 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.062721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.062957 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.063588 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:00.063604 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:00.064999 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:00.066978 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:00.066983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:00.066996 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:00.067395 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:00.067708 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:00.067714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:00.067716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:00.067953 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:00.068384 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:00.068392 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:00.068725 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.068964 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.069333 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
I20260619 22:25:00.071681 19644 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44793
I20260619 22:25:00.071964 19644 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:00.072611 19644 heartbeater.cc:511] Master 127.1.74.190:44793 requested a full tablet report, sending...
I20260619 22:25:00.074306 19453 ts_manager.cc:194] Registered new tserver with Master: fa803eef014e4e62a13c313e75ad0958 (127.1.74.129:43023)
I20260619 22:25:00.075526 19453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:42501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:25:00.086891 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44793
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:00.201762 19649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:00.202243 19649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:00.202292 19649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:00.206694 19649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:00.206768 19649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:00.206898 19649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:25:00.212534 19649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:00.213639 19649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:00.214670 19649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:25:00.224444 19649 server_base.cc:1034] running on GCE node
W20260619 22:25:00.224391 19655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:00.224395 19657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:00.224375 19654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:00.225121 19649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:00.225781 19649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:00.226945 19649 hybrid_clock.cc:648] HybridClock initialized: now 1781907900226908 us; error 48 us; skew 500 ppm
Jun 19 22:25:00 dist-test-slave-k84h krb5kdc[19408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907900, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:00.229755 19649 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:25:00.230700 19649 webserver.cc:460] Webserver started at http://127.1.74.130:33823/ using document root <none> and password file <none>
I20260619 22:25:00.231310 19649 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:00.231374 19649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:00.231585 19649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:00.233630 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "9da11f8dd7a84086b65b75ed14767145"
format_stamp: "Formatted at 2026-06-19 22:25:00 on dist-test-slave-k84h"
server_key: "14d1017dcaa3d8241c76d9956481e433"
server_key_iv: "1649858563624a25896c654b87651878"
server_key_version: "encryptionkey@0"
I20260619 22:25:00.234182 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "9da11f8dd7a84086b65b75ed14767145"
format_stamp: "Formatted at 2026-06-19 22:25:00 on dist-test-slave-k84h"
server_key: "14d1017dcaa3d8241c76d9956481e433"
server_key_iv: "1649858563624a25896c654b87651878"
server_key_version: "encryptionkey@0"
I20260619 22:25:00.237951 19649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:25:00.240731 19664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.242110 19649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:25:00.242251 19649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "9da11f8dd7a84086b65b75ed14767145"
format_stamp: "Formatted at 2026-06-19 22:25:00 on dist-test-slave-k84h"
server_key: "14d1017dcaa3d8241c76d9956481e433"
server_key_iv: "1649858563624a25896c654b87651878"
server_key_version: "encryptionkey@0"
I20260619 22:25:00.242377 19649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:00.264700 19649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:00.267854 19649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:00.268069 19649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:00.268671 19649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:00.269512 19649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:00.269583 19649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.269658 19649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:00.269693 19649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.278672 19649 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:46295
I20260619 22:25:00.278697 19777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:46295 every 8 connection(s)
I20260619 22:25:00.279635 19649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:25:00.284509 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19649
I20260619 22:25:00.284664 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:25:00.284942 1322 external_mini_cluster.cc:1428] Setting key 3efb2b57e089f20e365cf3bf4eabce19
Jun 19 22:25:00 dist-test-slave-k84h krb5kdc[19408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907900, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
I20260619 22:25:00.296836 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.281658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:45607 (local address 127.1.74.190:44793)
0619 22:25:00.281808 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.281812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.282575 (+ 763us) server_negotiation.cc:407] Connection header received
0619 22:25:00.284153 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.284157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.284264 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.284381 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:00.285809 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.286441 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.288441 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.288882 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.289013 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.292248 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:00.292255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:00.292261 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:00.292306 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:00.293920 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.294731 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.294734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.294735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.294817 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.295571 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.295574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.295575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.295772 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:00.295850 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.296588 (+ 738us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.296703 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:25:00.297863 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.281934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44793 (local address 127.1.74.130:45607)
0619 22:25:00.282412 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.282458 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.283915 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.284568 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.284575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.285112 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:00.285674 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.285696 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.286573 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.286579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.288280 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.288296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.289059 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.289062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.289227 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.289947 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:00.289965 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:00.291973 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:00.294100 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:00.294110 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:00.294135 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:00.294586 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:00.295198 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:00.295201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:00.295205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:00.295457 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:00.295899 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:00.295909 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:00.296330 (+ 421us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.296524 (+ 194us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.296822 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:25:00.299315 19778 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44793
I20260619 22:25:00.299609 19778 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:00.300143 19778 heartbeater.cc:511] Master 127.1.74.190:44793 requested a full tablet report, sending...
I20260619 22:25:00.301306 19453 ts_manager.cc:194] Registered new tserver with Master: 9da11f8dd7a84086b65b75ed14767145 (127.1.74.130:46295)
I20260619 22:25:00.301920 19453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:45607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:25:00.318997 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44793
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:00.435986 19783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:00.436268 19783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:00.436319 19783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:00.440469 19783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:00.440610 19783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:00.440732 19783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:25:00.446285 19783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:00.447820 19783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:00.448855 19783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:00.458868 19789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:00.458920 19788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:00.459005 19783 server_base.cc:1034] running on GCE node
W20260619 22:25:00.459178 19791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:00.459589 19783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:00.460278 19783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:00.461469 19783 hybrid_clock.cc:648] HybridClock initialized: now 1781907900461434 us; error 47 us; skew 500 ppm
Jun 19 22:25:00 dist-test-slave-k84h krb5kdc[19408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907900, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:00.464313 19783 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:25:00.465255 19783 webserver.cc:460] Webserver started at http://127.1.74.131:44209/ using document root <none> and password file <none>
I20260619 22:25:00.465854 19783 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:00.465956 19783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:00.466220 19783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:00.468429 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "87ac5872a2d842099ffcb6cc2b6058f5"
format_stamp: "Formatted at 2026-06-19 22:25:00 on dist-test-slave-k84h"
server_key: "74a225e4c557d299feb8b46cd81f8286"
server_key_iv: "2d60c45550cc4c3fa70bc479db5d76b2"
server_key_version: "encryptionkey@0"
I20260619 22:25:00.469015 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "87ac5872a2d842099ffcb6cc2b6058f5"
format_stamp: "Formatted at 2026-06-19 22:25:00 on dist-test-slave-k84h"
server_key: "74a225e4c557d299feb8b46cd81f8286"
server_key_iv: "2d60c45550cc4c3fa70bc479db5d76b2"
server_key_version: "encryptionkey@0"
I20260619 22:25:00.472870 19783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:00.475396 19798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.476747 19783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:00.476904 19783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "87ac5872a2d842099ffcb6cc2b6058f5"
format_stamp: "Formatted at 2026-06-19 22:25:00 on dist-test-slave-k84h"
server_key: "74a225e4c557d299feb8b46cd81f8286"
server_key_iv: "2d60c45550cc4c3fa70bc479db5d76b2"
server_key_version: "encryptionkey@0"
I20260619 22:25:00.477034 19783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:00.505530 19783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:00.509030 19783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:00.509241 19783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:00.509843 19783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:00.510702 19783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:00.510772 19783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.510885 19783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:00.510941 19783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:00.520105 19783 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:46601
I20260619 22:25:00.520138 19911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:46601 every 8 connection(s)
I20260619 22:25:00.521091 19783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:25:00.526330 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19783
I20260619 22:25:00.526480 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:25:00.526830 1322 external_mini_cluster.cc:1428] Setting key 5e880fceef7df8b3d4929e46f235a8ac
Jun 19 22:25:00 dist-test-slave-k84h krb5kdc[19408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907900, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:00.535445 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.523020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:42109 (local address 127.1.74.190:44793)
0619 22:25:00.523177 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.523181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.524022 (+ 841us) server_negotiation.cc:407] Connection header received
0619 22:25:00.525242 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.525244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.525311 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.525384 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:00.526699 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.527468 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.529058 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.529307 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.529422 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.532052 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:00.532056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:00.532059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:00.532092 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:00.533357 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.533861 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.533864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.533866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.533950 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.534289 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.534294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.534295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.534485 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:00.534598 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.535155 (+ 557us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.535280 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 22:25:00.536278 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.523408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44793 (local address 127.1.74.131:42109)
0619 22:25:00.523862 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.523906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.525053 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.525631 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.525638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.526017 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:00.526537 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.526557 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.527629 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.527632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.528930 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.528939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.529457 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.529461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.529634 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.530268 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:00.530284 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:00.531868 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:00.533481 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:00.533487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:00.533502 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:00.533756 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:00.534050 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:00.534053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:00.534055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:00.534194 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:00.534598 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:00.534606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:00.534912 (+ 306us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.535103 (+ 191us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.535443 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":111,"threads_started":1}
I20260619 22:25:00.537479 19912 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44793
I20260619 22:25:00.537729 19912 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:00.538187 19912 heartbeater.cc:511] Master 127.1.74.190:44793 requested a full tablet report, sending...
I20260619 22:25:00.539184 19453 ts_manager.cc:194] Registered new tserver with Master: 87ac5872a2d842099ffcb6cc2b6058f5 (127.1.74.131:46601)
I20260619 22:25:00.539850 19453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:42109
I20260619 22:25:00.540877 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:25:00.550606 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.543018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.1.74.190:44793)
0619 22:25:00.543160 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.543166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.543355 (+ 189us) server_negotiation.cc:407] Connection header received
0619 22:25:00.543534 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.543542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.543579 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.543657 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:00.544352 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.544845 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.545678 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.545890 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.546014 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.547590 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:00.547593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:00.547597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:00.547629 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:00.548799 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.549294 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.549297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.549299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.549377 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:00.549699 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:00.549702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:00.549704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:00.549895 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:00.550066 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.550337 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.550429 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:25:00.553843 19453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:25:00.557089 19453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:25:00.571370 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.565861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:43023 (local address 127.0.0.1:54762)
0619 22:25:00.566554 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.566571 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.566669 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.567340 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.567343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.567362 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:00.567478 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.567488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.568751 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.568757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.569787 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.569796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.570799 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.570803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.570940 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.570992 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.571095 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.571220 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":77,"thread_start_us":136,"threads_started":1}
I20260619 22:25:00.571379 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.566482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.1.74.129:43023)
0619 22:25:00.566923 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.566936 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.566959 (+ 23us) server_negotiation.cc:407] Connection header received
0619 22:25:00.567036 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.567041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.567214 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.567360 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:00.567617 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.568631 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.569926 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.570675 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.570798 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.571090 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.571142 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.571229 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":102,"threads_started":1}
I20260619 22:25:00.571679 19923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.565688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46601 (local address 127.0.0.1:33824)
0619 22:25:00.566206 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.566255 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.566382 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.567175 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.567181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.567202 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:00.567353 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.567361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.568818 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.568821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.570013 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.570021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.571251 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.571254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.571349 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.571371 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.571439 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.571519 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":23,"thread_start_us":87,"threads_started":1}
I20260619 22:25:00.572208 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.565853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.1.74.131:46601)
0619 22:25:00.566346 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.566359 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.566387 (+ 28us) server_negotiation.cc:407] Connection header received
0619 22:25:00.566599 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.566607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.566805 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.566991 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:00.567488 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.568670 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.570146 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.570976 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.571073 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.571197 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.571970 (+ 773us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.572069 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":161,"threads_started":1}
I20260619 22:25:00.573429 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.566420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:46295 (local address 127.0.0.1:34328)
0619 22:25:00.567608 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.567621 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.567711 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.568126 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.568129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.568147 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:00.568238 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.568244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.569550 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.569554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.571588 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.571597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.573036 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.573040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.573136 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.573158 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.573228 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.573296 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":88,"threads_started":1}
I20260619 22:25:00.573822 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.566831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.1.74.130:46295)
0619 22:25:00.567248 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.567261 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.567719 (+ 458us) server_negotiation.cc:407] Connection header received
0619 22:25:00.567805 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.567809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.567972 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.568115 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:00.568353 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.569390 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.571715 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.572741 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.572844 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.572973 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.573595 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.573687 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":124,"threads_started":1}
I20260619 22:25:00.574764 19846 tablet_service.cc:1467] Processing CreateTablet for tablet d1fb4235c10f4f018b615940dcebba83 (DEFAULT_TABLE table=test-table [id=c1e27f8c6814411cb27f13efc10a9426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:00.574759 19578 tablet_service.cc:1467] Processing CreateTablet for tablet d1fb4235c10f4f018b615940dcebba83 (DEFAULT_TABLE table=test-table [id=c1e27f8c6814411cb27f13efc10a9426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:00.575902 19846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1fb4235c10f4f018b615940dcebba83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:00.575919 19578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1fb4235c10f4f018b615940dcebba83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:00.577586 19712 tablet_service.cc:1467] Processing CreateTablet for tablet d1fb4235c10f4f018b615940dcebba83 (DEFAULT_TABLE table=test-table [id=c1e27f8c6814411cb27f13efc10a9426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:00.578658 19712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1fb4235c10f4f018b615940dcebba83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:00.583698 19930 tablet_bootstrap.cc:492] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Bootstrap starting.
I20260619 22:25:00.586333 19930 tablet_bootstrap.cc:654] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:00.587492 19929 tablet_bootstrap.cc:492] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: Bootstrap starting.
I20260619 22:25:00.587697 19930 log.cc:824] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:00.587759 19931 tablet_bootstrap.cc:492] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: Bootstrap starting.
I20260619 22:25:00.589648 19931 tablet_bootstrap.cc:654] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:00.589733 19929 tablet_bootstrap.cc:654] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:00.590025 19930 tablet_bootstrap.cc:492] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: No bootstrap required, opened a new log
I20260619 22:25:00.590229 19930 ts_tablet_manager.cc:1397] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 22:25:00.590592 19929 log.cc:824] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:00.590772 19931 log.cc:824] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:00.592615 19929 tablet_bootstrap.cc:492] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: No bootstrap required, opened a new log
I20260619 22:25:00.592837 19929 ts_tablet_manager.cc:1397] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:25:00.593098 19931 tablet_bootstrap.cc:492] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: No bootstrap required, opened a new log
I20260619 22:25:00.593277 19931 ts_tablet_manager.cc:1397] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260619 22:25:00.594282 19930 raft_consensus.cc:348] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.594574 19930 raft_consensus.cc:374] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:00.594631 19930 raft_consensus.cc:729] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ac5872a2d842099ffcb6cc2b6058f5, State: Initialized, Role: FOLLOWER
I20260619 22:25:00.595134 19930 consensus_queue.cc:260] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.595868 19912 heartbeater.cc:503] Master 127.1.74.190:44793 was elected leader, sending a full tablet report...
I20260619 22:25:00.596231 19930 ts_tablet_manager.cc:1428] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 22:25:00.597011 19929 raft_consensus.cc:348] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.597307 19929 raft_consensus.cc:374] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:00.597373 19929 raft_consensus.cc:729] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa803eef014e4e62a13c313e75ad0958, State: Initialized, Role: FOLLOWER
I20260619 22:25:00.597208 19931 raft_consensus.cc:348] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.597512 19931 raft_consensus.cc:374] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:00.597573 19931 raft_consensus.cc:729] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da11f8dd7a84086b65b75ed14767145, State: Initialized, Role: FOLLOWER
I20260619 22:25:00.597883 19929 consensus_queue.cc:260] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.598044 19931 consensus_queue.cc:260] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.598842 19778 heartbeater.cc:503] Master 127.1.74.190:44793 was elected leader, sending a full tablet report...
I20260619 22:25:00.599115 19931 ts_tablet_manager.cc:1428] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260619 22:25:00.599133 19929 ts_tablet_manager.cc:1428] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260619 22:25:00.599480 19644 heartbeater.cc:503] Master 127.1.74.190:44793 was elected leader, sending a full tablet report...
W20260619 22:25:00.772157 19913 tablet.cc:2369] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:25:00.780956 19779 tablet.cc:2369] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:25:00.805879 19645 tablet.cc:2369] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:25:00.806032 19935 raft_consensus.cc:482] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:25:00.806249 19935 raft_consensus.cc:504] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.807538 19935 leader_election.cc:288] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa803eef014e4e62a13c313e75ad0958 (127.1.74.129:43023), 9da11f8dd7a84086b65b75ed14767145 (127.1.74.130:46295)
I20260619 22:25:00.811604 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.807895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:48719 (local address 127.1.74.130:46295)
0619 22:25:00.808023 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.808027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.808473 (+ 446us) server_negotiation.cc:407] Connection header received
0619 22:25:00.808625 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.808629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.808688 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.808772 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:00.809043 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.809575 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.810610 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.811017 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.811147 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.811202 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.811400 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.811480 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 22:25:00.811615 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.807795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:43023 (local address 127.1.74.131:55845)
0619 22:25:00.807950 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.807968 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.808067 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.808389 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.808391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.808444 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:00.808606 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.808613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.809501 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.809505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.810365 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.810381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.811177 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.811181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.811296 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.811312 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.811444 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.811509 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 22:25:00.811620 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.807794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:46295 (local address 127.1.74.131:48719)
0619 22:25:00.808351 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.808366 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:00.808507 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:00.808797 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:00.808799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:00.808816 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:00.808933 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.808939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.809689 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.809692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.810485 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:00.810494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.811140 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.811144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:00.811243 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.811265 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:25:00.811359 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 22:25:00.811452 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":152,"thread_start_us":129,"threads_started":1}
I20260619 22:25:00.811677 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.807918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:55845 (local address 127.1.74.129:43023)
0619 22:25:00.808031 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.808036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.808052 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:25:00.808189 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.808193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.808249 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.808339 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:00.808726 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.809385 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.810497 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.811034 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.811164 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.811216 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.811485 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.811558 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:25:00.812337 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fb4235c10f4f018b615940dcebba83" candidate_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa803eef014e4e62a13c313e75ad0958" is_pre_election: true
I20260619 22:25:00.812336 19732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fb4235c10f4f018b615940dcebba83" candidate_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da11f8dd7a84086b65b75ed14767145" is_pre_election: true
I20260619 22:25:00.812641 19598 raft_consensus.cc:2454] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87ac5872a2d842099ffcb6cc2b6058f5 in term 0.
I20260619 22:25:00.812659 19732 raft_consensus.cc:2454] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87ac5872a2d842099ffcb6cc2b6058f5 in term 0.
I20260619 22:25:00.813143 19801 leader_election.cc:302] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 87ac5872a2d842099ffcb6cc2b6058f5, 9da11f8dd7a84086b65b75ed14767145, fa803eef014e4e62a13c313e75ad0958; no voters:
I20260619 22:25:00.813427 19935 raft_consensus.cc:2781] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:25:00.813512 19935 raft_consensus.cc:482] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:25:00.813544 19935 raft_consensus.cc:3037] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:00.814595 19935 raft_consensus.cc:504] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.815009 19935 leader_election.cc:288] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [CANDIDATE]: Term 1 election: Requested vote from peers fa803eef014e4e62a13c313e75ad0958 (127.1.74.129:43023), 9da11f8dd7a84086b65b75ed14767145 (127.1.74.130:46295)
I20260619 22:25:00.815394 19598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fb4235c10f4f018b615940dcebba83" candidate_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa803eef014e4e62a13c313e75ad0958"
I20260619 22:25:00.815449 19732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1fb4235c10f4f018b615940dcebba83" candidate_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da11f8dd7a84086b65b75ed14767145"
I20260619 22:25:00.815513 19598 raft_consensus.cc:3037] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:00.815635 19732 raft_consensus.cc:3037] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:00.816504 19598 raft_consensus.cc:2454] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87ac5872a2d842099ffcb6cc2b6058f5 in term 1.
I20260619 22:25:00.816809 19799 leader_election.cc:302] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87ac5872a2d842099ffcb6cc2b6058f5, fa803eef014e4e62a13c313e75ad0958; no voters:
I20260619 22:25:00.816881 19732 raft_consensus.cc:2454] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87ac5872a2d842099ffcb6cc2b6058f5 in term 1.
I20260619 22:25:00.816973 19935 raft_consensus.cc:2781] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:00.817238 19935 raft_consensus.cc:686] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 1 LEADER]: Becoming Leader. State: Replica: 87ac5872a2d842099ffcb6cc2b6058f5, State: Running, Role: LEADER
I20260619 22:25:00.817605 19935 consensus_queue.cc:237] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } }
I20260619 22:25:00.820807 19453 catalog_manager.cc:5495] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87ac5872a2d842099ffcb6cc2b6058f5 (127.1.74.131). New cstate: current_term: 1 leader_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ac5872a2d842099ffcb6cc2b6058f5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 } health_report { overall_health: UNKNOWN } } }
I20260619 22:25:00.873611 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.869803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.1.74.131:46601)
0619 22:25:00.869931 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:00.869936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:00.869953 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:25:00.870096 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:00.870099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:00.870155 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:00.870280 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:00.870554 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.871105 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.871977 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:00.872192 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:00.872306 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:00.872650 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:00.873107 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:00.873276 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:00.873379 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 22:25:00.873421 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 22:25:00.880507 19598 raft_consensus.cc:1261] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958 [term 1 FOLLOWER]: Refusing update from remote peer 87ac5872a2d842099ffcb6cc2b6058f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:00.880507 19732 raft_consensus.cc:1261] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 1 FOLLOWER]: Refusing update from remote peer 87ac5872a2d842099ffcb6cc2b6058f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:00.881457 19939 consensus_queue.cc:1035] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa803eef014e4e62a13c313e75ad0958" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:00.881737 19935 consensus_queue.cc:1035] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da11f8dd7a84086b65b75ed14767145" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:00.889981 19943 mvcc.cc:205] Tried to move back new op lower bound from 7298694761997164544 to 7298694761751543808. Current Snapshot: MvccSnapshot[applied={T|T < 7298694761997164544}]
I20260619 22:25:00.891978 19944 mvcc.cc:205] Tried to move back new op lower bound from 7298694761997164544 to 7298694761751543808. Current Snapshot: MvccSnapshot[applied={T|T < 7298694761997164544}]
I20260619 22:25:00.899124 19453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:00.899358 19453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:00.902415 19453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dab987d44ba48f4a6f77baf4db5cc8f: Sending DeleteTablet for 3 replicas of tablet d1fb4235c10f4f018b615940dcebba83
I20260619 22:25:00.903542 19578 tablet_service.cc:1514] Processing DeleteTablet for tablet d1fb4235c10f4f018b615940dcebba83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:00 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:54762
I20260619 22:25:00.904189 19951 tablet_replica.cc:330] T d1fb4235c10f4f018b615940dcebba83 P fa803eef014e4e62a13c313e75ad0958: stopping tablet replica
I20260619 22:25:00.904206 19712 tablet_service.cc:1514] Processing DeleteTablet for tablet d1fb4235c10f4f018b615940dcebba83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:00 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:34328
I20260619 22:25:00.904335 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19515
I20260619 22:25:00.904711 19846 tablet_service.cc:1514] Processing DeleteTablet for tablet d1fb4235c10f4f018b615940dcebba83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:00 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:33824
I20260619 22:25:00.906186 19952 tablet_replica.cc:330] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145: stopping tablet replica
I20260619 22:25:00.906416 19953 tablet_replica.cc:330] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: stopping tablet replica
I20260619 22:25:00.906631 19952 raft_consensus.cc:2229] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:25:00.906944 19953 raft_consensus.cc:2229] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:25:00.907078 19952 raft_consensus.cc:2258] T d1fb4235c10f4f018b615940dcebba83 P 9da11f8dd7a84086b65b75ed14767145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:00.907516 19953 raft_consensus.cc:2258] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:00.909049 19953 ts_tablet_manager.cc:1905] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:25:00.911584 19953 ts_tablet_manager.cc:1918] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:25:00.911720 19953 log.cc:1196] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/d1fb4235c10f4f018b615940dcebba83
I20260619 22:25:00.912036 19953 ts_tablet_manager.cc:1939] T d1fb4235c10f4f018b615940dcebba83 P 87ac5872a2d842099ffcb6cc2b6058f5: Deleting consensus metadata
I20260619 22:25:00.913378 19439 catalog_manager.cc:4841] TS 87ac5872a2d842099ffcb6cc2b6058f5 (127.1.74.131:46601): tablet d1fb4235c10f4f018b615940dcebba83 (table test-table [id=c1e27f8c6814411cb27f13efc10a9426]) successfully deleted
W20260619 22:25:00.913802 19437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:43023 (error 108)
I20260619 22:25:00.913861 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19649
I20260619 22:25:00.915587 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.915253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:43023 (local address 127.0.0.1:54766)
0619 22:25:00.915395 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.915490 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:43023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 22:25:00.915767 19437 catalog_manager.cc:4584] TS fa803eef014e4e62a13c313e75ad0958 (127.1.74.129:43023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1fb4235c10f4f018b615940dcebba83: Network error: Client connection negotiation failed: client connection to 127.1.74.129:43023: connect: Connection refused (error 111)
I20260619 22:25:00.922011 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19783
I20260619 22:25:00.923238 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:00.922873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:46295 (local address 127.0.0.1:34340)
0619 22:25:00.923080 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:00.923138 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:46295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260619 22:25:00.928452 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19421
2026-06-19T22:25:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2568 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T22:25:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:00Z Disabled control of system clock
I20260619 22:25:00.963068 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44177
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 22:25:01.076838 19958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:01.077131 19958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:01.077167 19958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:01.081313 19958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:01.081416 19958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:01.081434 19958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:01.081452 19958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:01.081467 19958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:01.086490 19958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:44177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:01.087785 19958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:01.089196 19958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:01.098532 19966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:01.098542 19964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:01.098758 19958 server_base.cc:1034] running on GCE node
W20260619 22:25:01.098542 19963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:01.099792 19958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:01.101043 19958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:01.102268 19958 hybrid_clock.cc:648] HybridClock initialized: now 1781907901102213 us; error 71 us; skew 500 ppm
I20260619 22:25:01.104540 19958 webserver.cc:460] Webserver started at http://127.1.74.190:35367/ using document root <none> and password file <none>
I20260619 22:25:01.105195 19958 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:01.105298 19958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:01.105543 19958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:01.107914 19958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "7589f6aa4fe640b684ef6031aa117a4b"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "9039b192e5f7a3826104225253792d18"
server_key_iv: "e29aa66fce58af17131effeef8a68f9e"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.108541 19958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "7589f6aa4fe640b684ef6031aa117a4b"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "9039b192e5f7a3826104225253792d18"
server_key_iv: "e29aa66fce58af17131effeef8a68f9e"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.112823 19958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:01.116024 19972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.117458 19958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:25:01.117726 19958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "7589f6aa4fe640b684ef6031aa117a4b"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "9039b192e5f7a3826104225253792d18"
server_key_iv: "e29aa66fce58af17131effeef8a68f9e"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.117872 19958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:01.137816 19958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:01.138710 19958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:01.138986 19958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:01.147217 19958 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:44177
I20260619 22:25:01.147260 20024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:44177 every 8 connection(s)
I20260619 22:25:01.148286 19958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:01.150930 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 19958
I20260619 22:25:01.151170 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:01.151551 1322 external_mini_cluster.cc:1428] Setting key ba139bb8cfdd89a84b2e087879530732
I20260619 22:25:01.151808 20025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:01.159104 20025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b: Bootstrap starting.
I20260619 22:25:01.161473 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.153256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.1.74.190:44177)
0619 22:25:01.154010 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.154026 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.154061 (+ 35us) server_negotiation.cc:407] Connection header received
0619 22:25:01.154699 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.154715 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.155171 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.155456 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:01.155781 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.156725 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.157978 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.158482 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.158614 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.159578 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:01.159583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:01.159597 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:01.160040 (+ 443us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:01.160100 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:01.160122 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:01.160202 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.160401 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.160711 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":160,"threads_started":1}
I20260619 22:25:01.161722 20025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:01.162473 20025 log.cc:824] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:01.164539 20025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b: No bootstrap required, opened a new log
I20260619 22:25:01.168432 20025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7589f6aa4fe640b684ef6031aa117a4b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44177 } }
I20260619 22:25:01.168675 20025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:01.168708 20025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7589f6aa4fe640b684ef6031aa117a4b, State: Initialized, Role: FOLLOWER
I20260619 22:25:01.169160 20025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7589f6aa4fe640b684ef6031aa117a4b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44177 } }
I20260619 22:25:01.169281 20025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:01.169348 20025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:01.169472 20025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:01.170440 20025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7589f6aa4fe640b684ef6031aa117a4b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44177 } }
I20260619 22:25:01.170785 20025 leader_election.cc:302] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7589f6aa4fe640b684ef6031aa117a4b; no voters:
I20260619 22:25:01.171190 20025 leader_election.cc:288] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:01.171362 20030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:01.171723 20030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [term 1 LEADER]: Becoming Leader. State: Replica: 7589f6aa4fe640b684ef6031aa117a4b, State: Running, Role: LEADER
I20260619 22:25:01.172053 20030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7589f6aa4fe640b684ef6031aa117a4b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44177 } }
I20260619 22:25:01.172505 20025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:01.173905 20032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7589f6aa4fe640b684ef6031aa117a4b. Latest consensus state: current_term: 1 leader_uuid: "7589f6aa4fe640b684ef6031aa117a4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7589f6aa4fe640b684ef6031aa117a4b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44177 } } }
I20260619 22:25:01.173938 20031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7589f6aa4fe640b684ef6031aa117a4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7589f6aa4fe640b684ef6031aa117a4b" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44177 } } }
I20260619 22:25:01.174036 20032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:01.174044 20031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:01.174476 20039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:01.177176 20039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:01.183270 20039 catalog_manager.cc:1347] Generated new cluster ID: 667a970f82be4d59adc43605911779d3
I20260619 22:25:01.183357 20039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:01.208441 20039 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:01.209584 20039 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:01.222501 20039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b: Generated new TSK 0
I20260619 22:25:01.223372 20039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:01.229610 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44177
--builtin_ntp_servers=127.1.74.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 22:25:01.340178 20049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:01.340452 20049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:01.340493 20049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:01.344656 20049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:01.344718 20049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:01.344825 20049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:25:01.349532 20049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:01.350543 20049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:01.351913 20049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:01.361171 20054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:01.361238 20057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:01.361198 20055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:01.361395 20049 server_base.cc:1034] running on GCE node
I20260619 22:25:01.361843 20049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:01.362390 20049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:01.363564 20049 hybrid_clock.cc:648] HybridClock initialized: now 1781907901363541 us; error 36 us; skew 500 ppm
I20260619 22:25:01.365193 20049 webserver.cc:460] Webserver started at http://127.1.74.129:41715/ using document root <none> and password file <none>
I20260619 22:25:01.365689 20049 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:01.365751 20049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:01.365899 20049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:01.367897 20049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "00d7a079c99b4687a1ba95cd1cdd811f"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "6b9cc6dcb5da57a8cf12e7b101cfc4f3"
server_key_iv: "ba73cf4b099026183a3b558d84ec55dc"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.368374 20049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "00d7a079c99b4687a1ba95cd1cdd811f"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "6b9cc6dcb5da57a8cf12e7b101cfc4f3"
server_key_iv: "ba73cf4b099026183a3b558d84ec55dc"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.371896 20049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:25:01.374364 20063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.375634 20049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:01.375722 20049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "00d7a079c99b4687a1ba95cd1cdd811f"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "6b9cc6dcb5da57a8cf12e7b101cfc4f3"
server_key_iv: "ba73cf4b099026183a3b558d84ec55dc"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.375808 20049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:01.391993 20049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:01.392624 20049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:01.392772 20049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:01.393278 20049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:01.394141 20049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:01.394186 20049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.394225 20049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:01.394241 20049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.403672 20049 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:45137
I20260619 22:25:01.403693 20176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:45137 every 8 connection(s)
I20260619 22:25:01.404561 20049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:25:01.405869 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20049
I20260619 22:25:01.406020 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:25:01.406375 1322 external_mini_cluster.cc:1428] Setting key 41b6ecf69ff07d82e538cd9b2be5eed9
I20260619 22:25:01.408449 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44177
--builtin_ntp_servers=127.1.74.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 22:25:01.415371 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.406616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:56745 (local address 127.1.74.190:44177)
0619 22:25:01.406812 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.406816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.407781 (+ 965us) server_negotiation.cc:407] Connection header received
0619 22:25:01.409295 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.409302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.409365 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.409463 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:01.410682 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.411711 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.412894 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.413321 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.413458 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.414658 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:01.414664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:01.414669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:01.414690 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:01.414718 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:01.414731 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:01.414822 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.415058 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.415143 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:25:01.416167 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.406880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44177 (local address 127.1.74.129:56745)
0619 22:25:01.407611 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:01.407691 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:01.409090 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:01.409666 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:01.409673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:01.409759 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:01.410492 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.410514 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.411863 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.411866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.412752 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.412762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.413515 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.413518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.413679 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.414374 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:25:01.414379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:01.414420 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:25:01.414488 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:01.414831 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:01.414875 (+ 44us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:01.414882 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 22:25:01.415018 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 22:25:01.415255 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":152,"threads_started":1}
I20260619 22:25:01.417603 20177 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44177
I20260619 22:25:01.417870 20177 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:01.418408 20177 heartbeater.cc:511] Master 127.1.74.190:44177 requested a full tablet report, sending...
I20260619 22:25:01.419974 19989 ts_manager.cc:194] Registered new tserver with Master: 00d7a079c99b4687a1ba95cd1cdd811f (127.1.74.129:45137)
I20260619 22:25:01.421119 19989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.129:56745
W20260619 22:25:01.523496 20180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:01.523774 20180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:01.523813 20180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:01.528111 20180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:01.528173 20180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:01.528263 20180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:25:01.533061 20180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:01.534251 20180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:01.535697 20180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:01.545454 20188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:01.545394 20185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:01.545429 20180 server_base.cc:1034] running on GCE node
W20260619 22:25:01.545394 20186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:01.546077 20180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:01.546725 20180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:01.547886 20180 hybrid_clock.cc:648] HybridClock initialized: now 1781907901547873 us; error 28 us; skew 500 ppm
I20260619 22:25:01.549636 20180 webserver.cc:460] Webserver started at http://127.1.74.130:46423/ using document root <none> and password file <none>
I20260619 22:25:01.550179 20180 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:01.550236 20180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:01.550401 20180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:01.552505 20180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "7f75c85d930449fca211b682254b5ccb"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "7b5d67d2890f8bc0e334622444f6dc6c"
server_key_iv: "2292d7bf2a5be9e22dd42f1eb6e1b75d"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.553089 20180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "7f75c85d930449fca211b682254b5ccb"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "7b5d67d2890f8bc0e334622444f6dc6c"
server_key_iv: "2292d7bf2a5be9e22dd42f1eb6e1b75d"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.556906 20180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:01.559449 20194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.560627 20180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:01.560721 20180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "7f75c85d930449fca211b682254b5ccb"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "7b5d67d2890f8bc0e334622444f6dc6c"
server_key_iv: "2292d7bf2a5be9e22dd42f1eb6e1b75d"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.560878 20180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:01.586139 20180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:01.586922 20180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:01.587137 20180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:01.587759 20180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:01.588733 20180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:01.588780 20180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.588850 20180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:01.588888 20180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.598802 20180 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:36941
I20260619 22:25:01.598826 20307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:36941 every 8 connection(s)
I20260619 22:25:01.599840 20180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:25:01.605976 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20180
I20260619 22:25:01.606102 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:25:01.606441 1322 external_mini_cluster.cc:1428] Setting key 51774df8a325a1eac91e480e6edcf646
I20260619 22:25:01.608755 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44177
--builtin_ntp_servers=127.1.74.148:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 22:25:01.610128 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.601850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:35193 (local address 127.1.74.190:44177)
0619 22:25:01.602004 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.602007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.603147 (+ 1140us) server_negotiation.cc:407] Connection header received
0619 22:25:01.604610 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.604613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.604673 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.604797 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:01.605697 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.606359 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.607787 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.608088 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.608193 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.609622 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:01.609625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:01.609627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:01.609640 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:01.609660 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:01.609668 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:01.609740 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.609941 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.609995 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 22:25:01.611248 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.602237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44177 (local address 127.1.74.130:35193)
0619 22:25:01.602985 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:01.603028 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:01.604383 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:01.604968 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:01.604974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:01.605060 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:01.605550 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.605562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.606486 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.606494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.607620 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.607630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.608290 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.608295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.608523 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.609266 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:25:01.609270 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:01.609331 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:25:01.609428 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:01.609762 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:01.609774 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:01.609783 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 22:25:01.609928 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 22:25:01.610295 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260619 22:25:01.612665 20308 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44177
I20260619 22:25:01.612954 20308 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:01.613551 20308 heartbeater.cc:511] Master 127.1.74.190:44177 requested a full tablet report, sending...
I20260619 22:25:01.614799 19989 ts_manager.cc:194] Registered new tserver with Master: 7f75c85d930449fca211b682254b5ccb (127.1.74.130:36941)
I20260619 22:25:01.615566 19989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.130:35193
W20260619 22:25:01.726112 20311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:01.726456 20311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:01.726528 20311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:01.731309 20311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:01.731451 20311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:01.731546 20311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:25:01.736797 20311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:44177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:01.738001 20311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:01.739543 20311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:01.749708 20316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:01.749708 20317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:01.749724 20319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:01.750141 20311 server_base.cc:1034] running on GCE node
I20260619 22:25:01.750507 20311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:01.751269 20311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:01.752491 20311 hybrid_clock.cc:648] HybridClock initialized: now 1781907901752461 us; error 47 us; skew 500 ppm
I20260619 22:25:01.754601 20311 webserver.cc:460] Webserver started at http://127.1.74.131:43861/ using document root <none> and password file <none>
I20260619 22:25:01.755275 20311 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:01.755373 20311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:01.755609 20311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:01.757752 20311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "e44f05bbeec8497f91d825e900a79eaf"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "7b4dd655152eed9a1d729ae3710005b9"
server_key_iv: "1b962f47f29b84f58c9f4cd8bd3b487e"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.758342 20311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "e44f05bbeec8497f91d825e900a79eaf"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "7b4dd655152eed9a1d729ae3710005b9"
server_key_iv: "1b962f47f29b84f58c9f4cd8bd3b487e"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.762671 20311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 22:25:01.765420 20325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.766799 20311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:25:01.767040 20311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "e44f05bbeec8497f91d825e900a79eaf"
format_stamp: "Formatted at 2026-06-19 22:25:01 on dist-test-slave-k84h"
server_key: "7b4dd655152eed9a1d729ae3710005b9"
server_key_iv: "1b962f47f29b84f58c9f4cd8bd3b487e"
server_key_version: "encryptionkey@0"
I20260619 22:25:01.767189 20311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:01.797119 20311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:01.797873 20311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:01.798089 20311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:01.798707 20311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:01.799690 20311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:01.799782 20311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.799850 20311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:01.799896 20311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:01.809000 20311 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:34233
I20260619 22:25:01.809026 20438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:34233 every 8 connection(s)
I20260619 22:25:01.809930 20311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:25:01.817022 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20311
I20260619 22:25:01.817124 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:25:01.817444 1322 external_mini_cluster.cc:1428] Setting key 5167fc7f3f04c7b03758b0c95b2a2f93
I20260619 22:25:01.819770 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.812000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:40309 (local address 127.1.74.190:44177)
0619 22:25:01.812177 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.812181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.813202 (+ 1021us) server_negotiation.cc:407] Connection header received
0619 22:25:01.814605 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.814608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.814657 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.814735 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:01.815687 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.816216 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.817426 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.817735 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.817849 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.819188 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:01.819192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:01.819194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:01.819207 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:01.819232 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:01.819241 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:01.819322 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.819549 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.819614 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 22:25:01.820699 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.812385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:44177 (local address 127.1.74.131:40309)
0619 22:25:01.813040 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:01.813081 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:01.814371 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:01.814932 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:01.814939 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:01.815023 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:01.815556 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.815569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.816344 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.816346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.817305 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.817314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.817917 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.817921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.818111 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.818921 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:25:01.818923 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:01.818968 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:25:01.819042 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:01.819369 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:01.819377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:01.819382 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 22:25:01.819504 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 22:25:01.819855 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":107,"threads_started":1}
I20260619 22:25:01.821920 20439 heartbeater.cc:347] Connected to a master server at 127.1.74.190:44177
I20260619 22:25:01.822146 20439 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:01.822688 20439 heartbeater.cc:511] Master 127.1.74.190:44177 requested a full tablet report, sending...
I20260619 22:25:01.823783 19989 ts_manager.cc:194] Registered new tserver with Master: e44f05bbeec8497f91d825e900a79eaf (127.1.74.131:34233)
I20260619 22:25:01.824337 19989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.131:40309
I20260619 22:25:01.832367 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:25:01.838723 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.834514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.1.74.190:44177)
0619 22:25:01.834702 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.834707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.834874 (+ 167us) server_negotiation.cc:407] Connection header received
0619 22:25:01.835040 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.835043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.835097 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.835182 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:01.835474 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.836014 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.836885 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.837122 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.837241 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.838180 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:01.838183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:01.838186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:01.838200 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:01.838227 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:01.838236 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:01.838372 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.838503 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.838555 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 22:25:01.841866 19989 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:25:01.845072 19989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:25:01.860743 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.855809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.1.74.130:36941)
0619 22:25:01.856124 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.856136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.856159 (+ 23us) server_negotiation.cc:407] Connection header received
0619 22:25:01.856234 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.856240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.856620 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.857131 (+ 511us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:01.857285 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.857987 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.859141 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.859953 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.860135 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.860344 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.860448 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.860555 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":138,"threads_started":1}
I20260619 22:25:01.860742 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.854484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36941 (local address 127.0.0.1:44206)
0619 22:25:01.854951 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:01.855008 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:01.855157 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:01.856785 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:01.856788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:01.856806 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:01.856983 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.856989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.858117 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.858122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.859019 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.859028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.860133 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.860137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.860253 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.860294 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 22:25:01.860418 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 22:25:01.860551 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":7,"spinlock_wait_cycles":5632,"thread_start_us":138,"threads_started":1}
I20260619 22:25:01.861121 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.855424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.1.74.129:45137)
0619 22:25:01.855748 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.855763 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.855789 (+ 26us) server_negotiation.cc:407] Connection header received
0619 22:25:01.855869 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.855873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.856043 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.856209 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:01.856533 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.857678 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.859345 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.860151 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.860251 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.860444 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.860825 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.860933 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":105,"threads_started":1}
I20260619 22:25:01.861279 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.854592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45137 (local address 127.0.0.1:48246)
0619 22:25:01.855017 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:01.855032 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:01.855157 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:01.856200 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:01.856207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:01.856229 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:01.856384 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.856393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.857816 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.857819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.859205 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.859216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.860502 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.860505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.860593 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.860615 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:25:01.860685 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 22:25:01.860745 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":1,"spinlock_wait_cycles":3840,"thread_start_us":93,"threads_started":1}
I20260619 22:25:01.861944 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.855348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:34233 (local address 127.0.0.1:38876)
0619 22:25:01.855591 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:01.855607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:01.855697 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:01.856983 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:01.856986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:01.857013 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:01.857111 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.857117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.858489 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.858492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.859926 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:01.859936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.861501 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.861505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:01.861629 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.861658 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 22:25:01.861744 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 22:25:01.861822 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":104,"threads_started":1}
I20260619 22:25:01.862071 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:01.855550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.1.74.131:34233)
0619 22:25:01.856432 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:01.856444 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:01.856466 (+ 22us) server_negotiation.cc:407] Connection header received
0619 22:25:01.856537 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:01.856542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:01.856685 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:01.856818 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:01.857242 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.858352 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.860082 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:01.861390 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:01.861506 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:01.861745 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:01.861804 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 22:25:01.861900 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":164,"threads_started":1}
I20260619 22:25:01.864306 20111 tablet_service.cc:1467] Processing CreateTablet for tablet 874416c4917446de955db0a1c10212bf (DEFAULT_TABLE table=test-table [id=b2e6a5e151904cb3add9410c267daeff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:01.865368 20111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 874416c4917446de955db0a1c10212bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:01.865187 20373 tablet_service.cc:1467] Processing CreateTablet for tablet 874416c4917446de955db0a1c10212bf (DEFAULT_TABLE table=test-table [id=b2e6a5e151904cb3add9410c267daeff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:01.865939 20373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 874416c4917446de955db0a1c10212bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:01.868430 20242 tablet_service.cc:1467] Processing CreateTablet for tablet 874416c4917446de955db0a1c10212bf (DEFAULT_TABLE table=test-table [id=b2e6a5e151904cb3add9410c267daeff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:01.869603 20242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 874416c4917446de955db0a1c10212bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:01.872711 20456 tablet_bootstrap.cc:492] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: Bootstrap starting.
I20260619 22:25:01.874183 20457 tablet_bootstrap.cc:492] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Bootstrap starting.
I20260619 22:25:01.876307 20456 tablet_bootstrap.cc:654] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:01.876468 20457 tablet_bootstrap.cc:654] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:01.877143 20456 log.cc:824] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:01.877300 20457 log.cc:824] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:01.877857 20458 tablet_bootstrap.cc:492] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Bootstrap starting.
I20260619 22:25:01.879606 20457 tablet_bootstrap.cc:492] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: No bootstrap required, opened a new log
I20260619 22:25:01.879606 20456 tablet_bootstrap.cc:492] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: No bootstrap required, opened a new log
I20260619 22:25:01.879819 20457 ts_tablet_manager.cc:1397] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:25:01.879819 20456 ts_tablet_manager.cc:1397] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 22:25:01.880934 20458 tablet_bootstrap.cc:654] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:01.881706 20458 log.cc:824] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:01.883863 20458 tablet_bootstrap.cc:492] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: No bootstrap required, opened a new log
I20260619 22:25:01.884104 20458 ts_tablet_manager.cc:1397] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:25:01.884503 20457 raft_consensus.cc:348] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:01.884511 20456 raft_consensus.cc:348] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:01.884811 20456 raft_consensus.cc:374] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:01.884811 20457 raft_consensus.cc:374] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:01.884871 20457 raft_consensus.cc:729] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f05bbeec8497f91d825e900a79eaf, State: Initialized, Role: FOLLOWER
I20260619 22:25:01.884871 20456 raft_consensus.cc:729] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7a079c99b4687a1ba95cd1cdd811f, State: Initialized, Role: FOLLOWER
I20260619 22:25:01.885378 20457 consensus_queue.cc:260] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:01.885378 20456 consensus_queue.cc:260] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:01.886214 20439 heartbeater.cc:503] Master 127.1.74.190:44177 was elected leader, sending a full tablet report...
I20260619 22:25:01.886513 20177 heartbeater.cc:503] Master 127.1.74.190:44177 was elected leader, sending a full tablet report...
I20260619 22:25:01.886572 20456 ts_tablet_manager.cc:1428] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 22:25:01.887087 20457 ts_tablet_manager.cc:1428] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 22:25:01.889663 20458 raft_consensus.cc:348] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:01.889919 20458 raft_consensus.cc:374] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:01.889953 20458 raft_consensus.cc:729] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f75c85d930449fca211b682254b5ccb, State: Initialized, Role: FOLLOWER
I20260619 22:25:01.890486 20458 consensus_queue.cc:260] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:01.891316 20308 heartbeater.cc:503] Master 127.1.74.190:44177 was elected leader, sending a full tablet report...
I20260619 22:25:01.892788 20458 ts_tablet_manager.cc:1428] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
W20260619 22:25:01.905768 20178 tablet.cc:2369] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:25:02.060930 20440 tablet.cc:2369] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 22:25:02.101084 20309 tablet.cc:2369] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:25:02.134569 20463 raft_consensus.cc:482] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:25:02.134876 20463 raft_consensus.cc:504] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:02.136044 20463 leader_election.cc:288] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f75c85d930449fca211b682254b5ccb (127.1.74.130:36941), 00d7a079c99b4687a1ba95cd1cdd811f (127.1.74.129:45137)
I20260619 22:25:02.140012 20441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.136356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45137 (local address 127.1.74.131:53733)
0619 22:25:02.136531 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:02.136549 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:02.136649 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:02.136945 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:02.136949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:02.136967 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:02.137105 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:02.137112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.137859 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.137865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:02.138718 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:02.138728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.139561 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.139565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:02.139667 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:02.139696 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 22:25:02.139787 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 22:25:02.139862 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 22:25:02.140000 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.136523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:53733 (local address 127.1.74.129:45137)
0619 22:25:02.136677 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:02.136683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:02.136703 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:25:02.136767 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:02.136770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:02.136824 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:02.136933 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:02.137230 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.137746 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.138877 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.139416 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.139569 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:02.139619 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:02.139803 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 22:25:02.139871 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:25:02.140266 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.136599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:49955 (local address 127.1.74.130:36941)
0619 22:25:02.136738 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:02.136742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:02.137149 (+ 407us) server_negotiation.cc:407] Connection header received
0619 22:25:02.137294 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:02.137299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:02.137347 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:02.137431 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:02.137703 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.138182 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.139329 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.139728 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.139861 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:02.139905 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:02.140091 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 22:25:02.140158 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:25:02.140283 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.136356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36941 (local address 127.1.74.131:49955)
0619 22:25:02.137059 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:02.137070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:02.137184 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:02.137442 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:02.137445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:02.137460 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:02.137594 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:02.137599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.138294 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.138298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:02.139213 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:02.139223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.139853 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.139856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:02.139962 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:02.139982 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 22:25:02.140075 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 22:25:02.140139 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":370,"thread_start_us":130,"threads_started":1}
I20260619 22:25:02.140908 20262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "874416c4917446de955db0a1c10212bf" candidate_uuid: "e44f05bbeec8497f91d825e900a79eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f75c85d930449fca211b682254b5ccb" is_pre_election: true
I20260619 22:25:02.140939 20131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "874416c4917446de955db0a1c10212bf" candidate_uuid: "e44f05bbeec8497f91d825e900a79eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" is_pre_election: true
I20260619 22:25:02.141229 20131 raft_consensus.cc:2454] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f05bbeec8497f91d825e900a79eaf in term 0.
I20260619 22:25:02.141229 20262 raft_consensus.cc:2454] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f05bbeec8497f91d825e900a79eaf in term 0.
I20260619 22:25:02.141741 20326 leader_election.cc:302] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00d7a079c99b4687a1ba95cd1cdd811f, 7f75c85d930449fca211b682254b5ccb, e44f05bbeec8497f91d825e900a79eaf; no voters:
I20260619 22:25:02.142055 20463 raft_consensus.cc:2781] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:25:02.142160 20463 raft_consensus.cc:482] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:25:02.142238 20463 raft_consensus.cc:3037] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:02.143438 20463 raft_consensus.cc:504] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:02.143848 20463 leader_election.cc:288] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [CANDIDATE]: Term 1 election: Requested vote from peers 7f75c85d930449fca211b682254b5ccb (127.1.74.130:36941), 00d7a079c99b4687a1ba95cd1cdd811f (127.1.74.129:45137)
I20260619 22:25:02.144232 20131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "874416c4917446de955db0a1c10212bf" candidate_uuid: "e44f05bbeec8497f91d825e900a79eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7a079c99b4687a1ba95cd1cdd811f"
I20260619 22:25:02.144229 20262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "874416c4917446de955db0a1c10212bf" candidate_uuid: "e44f05bbeec8497f91d825e900a79eaf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f75c85d930449fca211b682254b5ccb"
I20260619 22:25:02.144347 20131 raft_consensus.cc:3037] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:02.144376 20262 raft_consensus.cc:3037] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:02.145516 20131 raft_consensus.cc:2454] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f05bbeec8497f91d825e900a79eaf in term 1.
I20260619 22:25:02.145529 20262 raft_consensus.cc:2454] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f05bbeec8497f91d825e900a79eaf in term 1.
I20260619 22:25:02.145874 20327 leader_election.cc:302] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7a079c99b4687a1ba95cd1cdd811f, e44f05bbeec8497f91d825e900a79eaf; no voters:
I20260619 22:25:02.146101 20463 raft_consensus.cc:2781] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:02.146395 20463 raft_consensus.cc:686] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 1 LEADER]: Becoming Leader. State: Replica: e44f05bbeec8497f91d825e900a79eaf, State: Running, Role: LEADER
I20260619 22:25:02.146757 20463 consensus_queue.cc:237] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } }
I20260619 22:25:02.150208 19989 catalog_manager.cc:5495] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf reported cstate change: term changed from 0 to 1, leader changed from <none> to e44f05bbeec8497f91d825e900a79eaf (127.1.74.131). New cstate: current_term: 1 leader_uuid: "e44f05bbeec8497f91d825e900a79eaf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f05bbeec8497f91d825e900a79eaf" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34233 } health_report { overall_health: HEALTHY } } }
I20260619 22:25:02.230993 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.227069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.1.74.131:34233)
0619 22:25:02.227199 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:02.227203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:02.227216 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:25:02.227286 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:02.227288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:02.227343 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:02.227434 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:02.227727 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.228239 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.229238 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:02.229463 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:02.229553 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:02.230002 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:02.230419 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:02.230531 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:02.230747 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 22:25:02.230782 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 22:25:02.237246 20262 raft_consensus.cc:1261] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 1 FOLLOWER]: Refusing update from remote peer e44f05bbeec8497f91d825e900a79eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:02.237246 20131 raft_consensus.cc:1261] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 1 FOLLOWER]: Refusing update from remote peer e44f05bbeec8497f91d825e900a79eaf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:02.238148 20466 consensus_queue.cc:1035] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f75c85d930449fca211b682254b5ccb" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 36941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:02.238395 20463 consensus_queue.cc:1035] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d7a079c99b4687a1ba95cd1cdd811f" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 45137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:02.256035 19989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:02.256238 19989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:02.258989 19989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7589f6aa4fe640b684ef6031aa117a4b: Sending DeleteTablet for 3 replicas of tablet 874416c4917446de955db0a1c10212bf
I20260619 22:25:02.259895 20111 tablet_service.cc:1514] Processing DeleteTablet for tablet 874416c4917446de955db0a1c10212bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:02 UTC) from {username='slave'} at 127.0.0.1:48246
I20260619 22:25:02.259892 20373 tablet_service.cc:1514] Processing DeleteTablet for tablet 874416c4917446de955db0a1c10212bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:02 UTC) from {username='slave'} at 127.0.0.1:38876
I20260619 22:25:02.260016 20242 tablet_service.cc:1514] Processing DeleteTablet for tablet 874416c4917446de955db0a1c10212bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:02 UTC) from {username='slave'} at 127.0.0.1:44206
I20260619 22:25:02.260447 20479 tablet_replica.cc:330] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f: stopping tablet replica
I20260619 22:25:02.260618 20481 tablet_replica.cc:330] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: stopping tablet replica
I20260619 22:25:02.260813 20479 raft_consensus.cc:2229] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:25:02.260977 20481 raft_consensus.cc:2229] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:25:02.261173 20479 raft_consensus.cc:2258] T 874416c4917446de955db0a1c10212bf P 00d7a079c99b4687a1ba95cd1cdd811f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:02.261245 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20049
I20260619 22:25:02.262663 20481 raft_consensus.cc:2258] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:02.262781 20480 tablet_replica.cc:330] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: stopping tablet replica
I20260619 22:25:02.263370 20480 raft_consensus.cc:2229] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:25:02.263758 20480 raft_consensus.cc:2258] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:02.265327 20480 ts_tablet_manager.cc:1905] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:25:02.266522 20481 ts_tablet_manager.cc:1905] T 874416c4917446de955db0a1c10212bf P 7f75c85d930449fca211b682254b5ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:25:02.268478 20480 ts_tablet_manager.cc:1918] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:25:02.268589 20480 log.cc:1196] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/874416c4917446de955db0a1c10212bf
W20260619 22:25:02.268774 19974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:45137 (error 108)
I20260619 22:25:02.268913 20480 ts_tablet_manager.cc:1939] T 874416c4917446de955db0a1c10212bf P e44f05bbeec8497f91d825e900a79eaf: Deleting consensus metadata
I20260619 22:25:02.269582 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20180
I20260619 22:25:02.270797 19973 catalog_manager.cc:4841] TS e44f05bbeec8497f91d825e900a79eaf (127.1.74.131:34233): tablet 874416c4917446de955db0a1c10212bf (table test-table [id=b2e6a5e151904cb3add9410c267daeff]) successfully deleted
I20260619 22:25:02.272414 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.272023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:45137 (local address 127.0.0.1:48260)
0619 22:25:02.272189 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:02.272300 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 22:25:02.272639 19974 catalog_manager.cc:4584] TS 00d7a079c99b4687a1ba95cd1cdd811f (127.1.74.129:45137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 874416c4917446de955db0a1c10212bf: Network error: Client connection negotiation failed: client connection to 127.1.74.129:45137: connect: Connection refused (error 111)
I20260619 22:25:02.276288 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20311
I20260619 22:25:02.278396 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:02.278035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:36941 (local address 127.0.0.1:44214)
0619 22:25:02.278202 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:02.278273 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:36941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 22:25:02.282130 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 19958
2026-06-19T22:25:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1350 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:02 dist-test-slave-k84h krb5kdc[20485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:02 dist-test-slave-k84h krb5kdc[20485](info): set up 2 sockets
Jun 19 22:25:02 dist-test-slave-k84h krb5kdc[20485](info): commencing operation
krb5kdc: starting...
W20260619 22:25:03.330943 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:03 dist-test-slave-k84h krb5kdc[20485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:03.462471 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33583
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:41371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:33583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:03.574702 20498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:03.575091 20498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:03.575183 20498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:03.579252 20498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:03.579358 20498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:03.579394 20498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:03.579432 20498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:03.579480 20498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:03.585105 20498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:41371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:33583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:03.586261 20498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:03.587313 20498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:03.595649 20504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:03.595834 20498 server_base.cc:1034] running on GCE node
W20260619 22:25:03.595620 20506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:03.595623 20503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:03.596490 20498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:03.597608 20498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:03.598829 20498 hybrid_clock.cc:648] HybridClock initialized: now 1781907903598820 us; error 33 us; skew 500 ppm
Jun 19 22:25:03 dist-test-slave-k84h krb5kdc[20485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:03.601846 20498 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:03.602937 20498 webserver.cc:460] Webserver started at http://127.1.74.190:41929/ using document root <none> and password file <none>
I20260619 22:25:03.603511 20498 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:03.603605 20498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:03.603857 20498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:03.605924 20498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "714fb60fb53b4d999374b08a284d5799"
format_stamp: "Formatted at 2026-06-19 22:25:03 on dist-test-slave-k84h"
server_key: "fea0e37044c72fd676e6729b0ded32e8"
server_key_iv: "c36f1886fb16afe337df9c909dadd6a2"
server_key_version: "encryptionkey@0"
I20260619 22:25:03.606514 20498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "714fb60fb53b4d999374b08a284d5799"
format_stamp: "Formatted at 2026-06-19 22:25:03 on dist-test-slave-k84h"
server_key: "fea0e37044c72fd676e6729b0ded32e8"
server_key_iv: "c36f1886fb16afe337df9c909dadd6a2"
server_key_version: "encryptionkey@0"
I20260619 22:25:03.610234 20498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 22:25:03.612866 20513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:03.614226 20498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:25:03.614398 20498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "714fb60fb53b4d999374b08a284d5799"
format_stamp: "Formatted at 2026-06-19 22:25:03 on dist-test-slave-k84h"
server_key: "fea0e37044c72fd676e6729b0ded32e8"
server_key_iv: "c36f1886fb16afe337df9c909dadd6a2"
server_key_version: "encryptionkey@0"
I20260619 22:25:03.614524 20498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:03.626044 20498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:03.629478 20498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:03.629690 20498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:03.637179 20498 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:33583
I20260619 22:25:03.637198 20565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:33583 every 8 connection(s)
I20260619 22:25:03.638197 20498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:03.639247 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20498
I20260619 22:25:03.639387 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:03.639665 1322 external_mini_cluster.cc:1428] Setting key d48ac95a6eed05fc5ccc58b127c718c2
I20260619 22:25:03.641389 20566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:03.648115 20566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799: Bootstrap starting.
Jun 19 22:25:03 dist-test-slave-k84h krb5kdc[20485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:03.650914 20566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:03.651921 20566 log.cc:824] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:03.654445 20566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799: No bootstrap required, opened a new log
I20260619 22:25:03.655494 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:03.641095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.1.74.190:33583)
0619 22:25:03.641507 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:03.641526 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:03.641567 (+ 41us) server_negotiation.cc:407] Connection header received
0619 22:25:03.642387 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:03.642408 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:03.642809 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:03.643205 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:03.643877 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:03.645395 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:03.646268 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:03.647021 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:03.647187 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:03.650054 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:03.650060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:03.650076 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:03.650109 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:03.652224 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:03.652796 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:03.652801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:03.652805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:03.652978 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:03.653261 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:03.653264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:03.653266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:03.653773 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:03.653951 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:03.654218 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 22:25:03.654520 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260619 22:25:03.658587 20566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714fb60fb53b4d999374b08a284d5799" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33583 } }
I20260619 22:25:03.658809 20566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:03.658891 20566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714fb60fb53b4d999374b08a284d5799, State: Initialized, Role: FOLLOWER
I20260619 22:25:03.659318 20566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714fb60fb53b4d999374b08a284d5799" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33583 } }
I20260619 22:25:03.659436 20566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:03.659475 20566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:03.659552 20566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:03.660452 20566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714fb60fb53b4d999374b08a284d5799" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33583 } }
I20260619 22:25:03.660754 20566 leader_election.cc:302] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 714fb60fb53b4d999374b08a284d5799; no voters:
I20260619 22:25:03.661015 20566 leader_election.cc:288] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:03.661278 20571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:03.661549 20571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [term 1 LEADER]: Becoming Leader. State: Replica: 714fb60fb53b4d999374b08a284d5799, State: Running, Role: LEADER
I20260619 22:25:03.661870 20571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714fb60fb53b4d999374b08a284d5799" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33583 } }
I20260619 22:25:03.662302 20566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:03.663678 20573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 714fb60fb53b4d999374b08a284d5799. Latest consensus state: current_term: 1 leader_uuid: "714fb60fb53b4d999374b08a284d5799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714fb60fb53b4d999374b08a284d5799" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33583 } } }
I20260619 22:25:03.663655 20572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "714fb60fb53b4d999374b08a284d5799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "714fb60fb53b4d999374b08a284d5799" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33583 } } }
I20260619 22:25:03.663803 20573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:03.663803 20572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:03.664212 20582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:03.667268 20582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:03.673408 20582 catalog_manager.cc:1347] Generated new cluster ID: 45d35080f9354b808af6c463c37a5762
I20260619 22:25:03.673516 20582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:03.687543 20582 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:03.688856 20582 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:03.699124 20582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799: Generated new TSK 0
I20260619 22:25:03.699923 20582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.74.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
Entry for principal kudu/127.1.74.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab.
I20260619 22:25:03.739327 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33583
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:41371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:03.855407 20592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:03.855695 20592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:03.855763 20592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:03.859992 20592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:03.860075 20592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:03.860170 20592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:25:03.866204 20592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:41371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.129.keytab
--principal=kudu/127.1.74.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:03.867441 20592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:03.868543 20592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:03.878568 20597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:03.878573 20600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:03.878588 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:03.878883 20592 server_base.cc:1034] running on GCE node
I20260619 22:25:03.879366 20592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:03.880079 20592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:03.881294 20592 hybrid_clock.cc:648] HybridClock initialized: now 1781907903881271 us; error 30 us; skew 500 ppm
Jun 19 22:25:03 dist-test-slave-k84h krb5kdc[20485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:03.884299 20592 init.cc:377] Logged in from keytab as kudu/127.1.74.129@KRBTEST.COM (short username kudu)
I20260619 22:25:03.885236 20592 webserver.cc:460] Webserver started at http://127.1.74.129:41617/ using document root <none> and password file <none>
I20260619 22:25:03.885835 20592 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:03.885897 20592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:03.886135 20592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:03.888278 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "cedf4b5b0bd249229a9808b1e0ba4e42"
format_stamp: "Formatted at 2026-06-19 22:25:03 on dist-test-slave-k84h"
server_key: "30b7b4e090d82f25d025c2c33fd36403"
server_key_iv: "36d1cbe5e5e557f0588ee2b69117c279"
server_key_version: "encryptionkey@0"
I20260619 22:25:03.888875 20592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "cedf4b5b0bd249229a9808b1e0ba4e42"
format_stamp: "Formatted at 2026-06-19 22:25:03 on dist-test-slave-k84h"
server_key: "30b7b4e090d82f25d025c2c33fd36403"
server_key_iv: "36d1cbe5e5e557f0588ee2b69117c279"
server_key_version: "encryptionkey@0"
I20260619 22:25:03.892750 20592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:03.895236 20607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:03.896474 20592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:25:03.896605 20592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "cedf4b5b0bd249229a9808b1e0ba4e42"
format_stamp: "Formatted at 2026-06-19 22:25:03 on dist-test-slave-k84h"
server_key: "30b7b4e090d82f25d025c2c33fd36403"
server_key_iv: "36d1cbe5e5e557f0588ee2b69117c279"
server_key_version: "encryptionkey@0"
I20260619 22:25:03.896720 20592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:03.924765 20592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:03.928323 20592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:03.928560 20592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:03.929222 20592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:03.930145 20592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:03.930199 20592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:03.930270 20592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:03.930315 20592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:03.940521 20592 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:41731
I20260619 22:25:03.940541 20720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:41731 every 8 connection(s)
I20260619 22:25:03.941534 20592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:25:03.946695 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20592
I20260619 22:25:03.946861 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:25:03.947218 1322 external_mini_cluster.cc:1428] Setting key 1a9d9ecabaf2050ffa0fe8e915f94e29
Jun 19 22:25:03 dist-test-slave-k84h krb5kdc[20485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907903, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.129@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.130@KRBTEST.COM; defaulting to no policy
I20260619 22:25:03.956580 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:03.943558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:54713 (local address 127.1.74.190:33583)
0619 22:25:03.943694 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:03.943698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:03.944602 (+ 904us) server_negotiation.cc:407] Connection header received
0619 22:25:03.945443 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:03.945446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:03.945501 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:03.945587 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:03.947126 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:03.948018 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:03.949540 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:03.949964 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:03.950093 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:03.952761 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:03.952765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:03.952768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:03.952803 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:03.954315 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:03.954934 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:03.954938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:03.954950 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:03.955032 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:03.955405 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:03.955408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:03.955410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:03.955613 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:03.955721 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:03.956261 (+ 540us) server_negotiation.cc:299] Negotiation successful
0619 22:25:03.956405 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 22:25:03.957520 20723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:03.943915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33583 (local address 127.1.74.129:54713)
0619 22:25:03.944406 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:03.944464 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:03.945234 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:03.945746 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:03.945752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:03.946150 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:03.946900 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:03.946924 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:03.948169 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:03.948172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:03.949395 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:03.949405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:03.950123 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:03.950126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:03.950280 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:03.950993 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:03.951010 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:03.952549 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:03.954467 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:03.954473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:03.954493 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:03.954781 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:03.955132 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:03.955135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:03.955137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:03.955298 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:03.955721 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:03.955729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:03.956007 (+ 278us) client_negotiation.cc:769] Sending connection context
0619 22:25:03.956205 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 22:25:03.956586 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":142,"threads_started":1}
I20260619 22:25:03.959066 20721 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33583
I20260619 22:25:03.959334 20721 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.130@KRBTEST.COM" created.
I20260619 22:25:03.959941 20721 heartbeater.cc:511] Master 127.1.74.190:33583 requested a full tablet report, sending...
I20260619 22:25:03.961572 20530 ts_manager.cc:194] Registered new tserver with Master: cedf4b5b0bd249229a9808b1e0ba4e42 (127.1.74.129:41731)
I20260619 22:25:03.963124 20530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.129@KRBTEST.COM'} at 127.1.74.129:54713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
Entry for principal kudu/127.1.74.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab.
I20260619 22:25:03.978574 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33583
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:41371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:04.093591 20726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:04.093915 20726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:04.093971 20726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:04.098328 20726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:04.098433 20726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:04.098497 20726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:25:04.104310 20726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:41371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.130.keytab
--principal=kudu/127.1.74.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:04.105535 20726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:04.106657 20726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:04.117488 20731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:04.117604 20732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:04.117488 20734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:04.117782 20726 server_base.cc:1034] running on GCE node
I20260619 22:25:04.118364 20726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:04.119213 20726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:04.120410 20726 hybrid_clock.cc:648] HybridClock initialized: now 1781907904120397 us; error 37 us; skew 500 ppm
Jun 19 22:25:04 dist-test-slave-k84h krb5kdc[20485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:04.123817 20726 init.cc:377] Logged in from keytab as kudu/127.1.74.130@KRBTEST.COM (short username kudu)
I20260619 22:25:04.124918 20726 webserver.cc:460] Webserver started at http://127.1.74.130:40107/ using document root <none> and password file <none>
I20260619 22:25:04.125525 20726 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:04.125620 20726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:04.125865 20726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:04.128122 20726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "67e8d847a2d3d4a97e2c2579072ea0be"
server_key_iv: "9aeaf042f0f20c4f2580df7b7175cdcc"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.128744 20726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "67e8d847a2d3d4a97e2c2579072ea0be"
server_key_iv: "9aeaf042f0f20c4f2580df7b7175cdcc"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.132773 20726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:04.135773 20741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.137126 20726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:04.137264 20726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "67e8d847a2d3d4a97e2c2579072ea0be"
server_key_iv: "9aeaf042f0f20c4f2580df7b7175cdcc"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.137395 20726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:04.179100 20726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:04.182582 20726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:04.182813 20726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:04.183513 20726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:04.184386 20726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:04.184432 20726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.184475 20726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:04.184530 20726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.195016 20726 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:33227
I20260619 22:25:04.195046 20854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:33227 every 8 connection(s)
I20260619 22:25:04.196054 20726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:25:04.196220 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20726
I20260619 22:25:04.196313 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:25:04.196614 1322 external_mini_cluster.cc:1428] Setting key 4dc2f26d88f9fe8354060f532d048a94
WARNING: no policy specified for kudu/127.1.74.131@KRBTEST.COM; defaulting to no policy
Jun 19 22:25:04 dist-test-slave-k84h krb5kdc[20485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.130@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.131@KRBTEST.COM" created.
I20260619 22:25:04.212113 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.198299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:58741 (local address 127.1.74.190:33583)
0619 22:25:04.198455 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.198459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.199378 (+ 919us) server_negotiation.cc:407] Connection header received
0619 22:25:04.200555 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.200562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.200631 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.200733 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:04.202471 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.203082 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.205024 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.205257 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.205338 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.208303 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:04.208306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:04.208308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:04.208333 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:04.209920 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:04.210490 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:04.210494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:04.210496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:04.210581 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:04.210951 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:04.210955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:04.210957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:04.211158 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:04.211265 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.211801 (+ 536us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.211929 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:25:04.212956 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.198618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33583 (local address 127.1.74.130:58741)
0619 22:25:04.199162 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.199264 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.200318 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.200962 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.200970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.201557 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:04.202267 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.202288 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.203218 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.203224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.204858 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.204874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.205440 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.205445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.205667 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.206336 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:04.206359 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:04.208128 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:04.210041 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:04.210048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:04.210066 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:04.210380 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:04.210682 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:04.210684 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:04.210687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:04.210828 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:04.211274 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:04.211282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:04.211550 (+ 268us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.211743 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.212084 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":158,"threads_started":1}
I20260619 22:25:04.214179 20855 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33583
I20260619 22:25:04.214489 20855 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:04.215175 20855 heartbeater.cc:511] Master 127.1.74.190:33583 requested a full tablet report, sending...
I20260619 22:25:04.216501 20530 ts_manager.cc:194] Registered new tserver with Master: 7bd7dc603eaa47c7bd0c88606ee4c6f3 (127.1.74.130:33227)
I20260619 22:25:04.217343 20530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.130@KRBTEST.COM'} at 127.1.74.130:58741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
Entry for principal kudu/127.1.74.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab.
I20260619 22:25:04.231110 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33583
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:41371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:04.345170 20860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:04.345507 20860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:04.345561 20860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:04.349813 20860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:04.349911 20860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:04.349977 20860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:25:04.355625 20860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:41371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.131.keytab
--principal=kudu/127.1.74.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:33583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:04.356750 20860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:04.357784 20860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:04.368076 20868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:04.368053 20866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:04.368130 20865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:04.368731 20860 server_base.cc:1034] running on GCE node
I20260619 22:25:04.369189 20860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:04.369915 20860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:04.371093 20860 hybrid_clock.cc:648] HybridClock initialized: now 1781907904371067 us; error 51 us; skew 500 ppm
Jun 19 22:25:04 dist-test-slave-k84h krb5kdc[20485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:04.373871 20860 init.cc:377] Logged in from keytab as kudu/127.1.74.131@KRBTEST.COM (short username kudu)
I20260619 22:25:04.374884 20860 webserver.cc:460] Webserver started at http://127.1.74.131:39703/ using document root <none> and password file <none>
I20260619 22:25:04.375501 20860 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:04.375597 20860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:04.375830 20860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:04.377955 20860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "d32f5ff813b6494e93d77511b3edef00"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "defdaa76003f3606e21ad74407914096"
server_key_iv: "a35859657b2f972a631410e015c32177"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.378566 20860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "d32f5ff813b6494e93d77511b3edef00"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "defdaa76003f3606e21ad74407914096"
server_key_iv: "a35859657b2f972a631410e015c32177"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.382596 20860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 22:25:04.385435 20875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.386906 20860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:25:04.387081 20860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "d32f5ff813b6494e93d77511b3edef00"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "defdaa76003f3606e21ad74407914096"
server_key_iv: "a35859657b2f972a631410e015c32177"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.387218 20860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:04.411037 20860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:04.414150 20860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:04.414387 20860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:04.415081 20860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:04.415941 20860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:04.416016 20860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.416100 20860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:04.416153 20860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.425519 20860 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:46415
I20260619 22:25:04.425535 20988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:46415 every 8 connection(s)
I20260619 22:25:04.426471 20860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:25:04.428180 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 20860
I20260619 22:25:04.428334 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:25:04.428653 1322 external_mini_cluster.cc:1428] Setting key f4d7805c2a151c2cc830fd6e2dbb6abc
Jun 19 22:25:04 dist-test-slave-k84h krb5kdc[20485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907904, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.131@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:04.440580 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.428295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:51325 (local address 127.1.74.190:33583)
0619 22:25:04.428472 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.428477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.429458 (+ 981us) server_negotiation.cc:407] Connection header received
0619 22:25:04.430568 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.430571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.430622 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.430710 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:04.431908 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.432449 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.433689 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.433905 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.433988 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.436751 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:04.436755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:04.436758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:04.436795 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:04.438275 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:04.438876 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:04.438879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:04.438881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:04.438966 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:04.439337 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:04.439341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:04.439343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:04.439548 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:04.439656 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.440265 (+ 609us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.440406 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 22:25:04.442066 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.428709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:33583 (local address 127.1.74.131:51325)
0619 22:25:04.429272 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.429343 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.430365 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.430872 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.430879 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.431248 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:04.431740 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.431752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.432565 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.432568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.433468 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.433477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.434043 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.434045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.434197 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.434872 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:04.434891 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:04.436551 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:04.438417 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:04.438423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:04.438441 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:04.438728 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:04.439067 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:04.439070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:04.439072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:04.439226 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:04.439654 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:04.439662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:04.440002 (+ 340us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.440214 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.440593 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260619 22:25:04.443437 20989 heartbeater.cc:347] Connected to a master server at 127.1.74.190:33583
I20260619 22:25:04.443650 20989 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:04.444110 20989 heartbeater.cc:511] Master 127.1.74.190:33583 requested a full tablet report, sending...
I20260619 22:25:04.445092 20530 ts_manager.cc:194] Registered new tserver with Master: d32f5ff813b6494e93d77511b3edef00 (127.1.74.131:46415)
I20260619 22:25:04.445662 20530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.74.131@KRBTEST.COM'} at 127.1.74.131:51325
I20260619 22:25:04.454756 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:25:04.464419 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.456839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.1.74.190:33583)
0619 22:25:04.456992 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.456997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.457185 (+ 188us) server_negotiation.cc:407] Connection header received
0619 22:25:04.457274 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.457277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.457319 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.457396 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:04.458086 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.458702 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.459575 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.459800 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.459900 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.461218 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:04.461221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:04.461223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:04.461245 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:04.462736 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:04.463234 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:04.463237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:04.463238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:04.463316 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:04.463619 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:04.463621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:04.463622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:04.463859 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:04.464007 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.464202 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.464295 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 22:25:04.467521 20530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:25:04.470387 20530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:25:04.484980 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.479486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.1.74.130:33227)
0619 22:25:04.479899 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.479911 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.480078 (+ 167us) server_negotiation.cc:407] Connection header received
0619 22:25:04.480283 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.480287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.480432 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.480544 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:04.481005 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.482093 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.483654 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.484245 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.484375 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.484533 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.484708 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.484807 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":112,"threads_started":1}
I20260619 22:25:04.485033 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.478895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46415 (local address 127.0.0.1:37324)
0619 22:25:04.479705 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.479719 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.479813 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.480712 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.480716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.480736 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:04.480870 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.480876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.481915 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.481919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.483493 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.483505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.484652 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.484656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.484727 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.484757 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.484817 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.484874 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":49,"spinlock_wait_cycles":1408,"thread_start_us":113,"threads_started":1}
I20260619 22:25:04.485194 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.478908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.1.74.129:41731)
0619 22:25:04.479499 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.479507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.479535 (+ 28us) server_negotiation.cc:407] Connection header received
0619 22:25:04.479762 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.479771 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.480009 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.480152 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:04.480647 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.481794 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.483370 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.484252 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.484367 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.484533 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.484952 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.485043 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":79,"threads_started":1}
I20260619 22:25:04.485458 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.478669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:41731 (local address 127.0.0.1:42150)
0619 22:25:04.479359 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.479414 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.479543 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.480329 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.480335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.480363 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:04.480512 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.480520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.481943 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.481947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.483225 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.483234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.484598 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.484602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.484692 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.484714 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.484785 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.484874 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":67,"thread_start_us":95,"threads_started":1}
I20260619 22:25:04.485616 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.479107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:33227 (local address 127.0.0.1:47784)
0619 22:25:04.479988 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.480004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.480146 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.480734 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.480737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.480748 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:04.480870 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.480876 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.482242 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.482246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.483493 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.483505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.484378 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.484382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.484489 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.484526 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.484676 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.484812 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":109,"spinlock_wait_cycles":9472,"thread_start_us":89,"threads_started":1}
I20260619 22:25:04.486555 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.479506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.1.74.131:46415)
0619 22:25:04.480109 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.480121 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.480140 (+ 19us) server_negotiation.cc:407] Connection header received
0619 22:25:04.480217 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.480225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.480402 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.480544 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:04.481027 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.481755 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.483615 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.484397 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.484491 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.484604 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.485445 (+ 841us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.485556 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":129,"threads_started":1}
I20260619 22:25:04.488754 20923 tablet_service.cc:1467] Processing CreateTablet for tablet e692b5e918194fe4b0658f6bc0fba2d2 (DEFAULT_TABLE table=test-table [id=fdc06ae0ed5341399b0dbda5c0f581b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:04.488765 20655 tablet_service.cc:1467] Processing CreateTablet for tablet e692b5e918194fe4b0658f6bc0fba2d2 (DEFAULT_TABLE table=test-table [id=fdc06ae0ed5341399b0dbda5c0f581b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:04.488765 20789 tablet_service.cc:1467] Processing CreateTablet for tablet e692b5e918194fe4b0658f6bc0fba2d2 (DEFAULT_TABLE table=test-table [id=fdc06ae0ed5341399b0dbda5c0f581b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:04.489801 20923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e692b5e918194fe4b0658f6bc0fba2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:04.489817 20655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e692b5e918194fe4b0658f6bc0fba2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:04.489817 20789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e692b5e918194fe4b0658f6bc0fba2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:04.496879 21008 tablet_bootstrap.cc:492] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: Bootstrap starting.
I20260619 22:25:04.497835 21006 tablet_bootstrap.cc:492] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Bootstrap starting.
I20260619 22:25:04.499126 21007 tablet_bootstrap.cc:492] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: Bootstrap starting.
I20260619 22:25:04.499733 21008 tablet_bootstrap.cc:654] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:04.499964 21006 tablet_bootstrap.cc:654] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:04.500710 21008 log.cc:824] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:04.500963 21007 tablet_bootstrap.cc:654] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:04.501093 21006 log.cc:824] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:04.501745 21007 log.cc:824] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:04.502844 21008 tablet_bootstrap.cc:492] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: No bootstrap required, opened a new log
I20260619 22:25:04.503039 21006 tablet_bootstrap.cc:492] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: No bootstrap required, opened a new log
I20260619 22:25:04.503073 21008 ts_tablet_manager.cc:1397] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260619 22:25:04.503216 21006 ts_tablet_manager.cc:1397] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 22:25:04.503715 21007 tablet_bootstrap.cc:492] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: No bootstrap required, opened a new log
I20260619 22:25:04.503877 21007 ts_tablet_manager.cc:1397] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 22:25:04.506461 21006 raft_consensus.cc:348] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.506713 21006 raft_consensus.cc:374] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:04.506763 21006 raft_consensus.cc:729] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32f5ff813b6494e93d77511b3edef00, State: Initialized, Role: FOLLOWER
I20260619 22:25:04.507223 21006 consensus_queue.cc:260] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.507112 21008 raft_consensus.cc:348] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.507365 21008 raft_consensus.cc:374] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:04.507421 21008 raft_consensus.cc:729] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd7dc603eaa47c7bd0c88606ee4c6f3, State: Initialized, Role: FOLLOWER
I20260619 22:25:04.507844 20989 heartbeater.cc:503] Master 127.1.74.190:33583 was elected leader, sending a full tablet report...
I20260619 22:25:04.507833 21008 consensus_queue.cc:260] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.508203 21006 ts_tablet_manager.cc:1428] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 22:25:04.508701 21008 ts_tablet_manager.cc:1428] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260619 22:25:04.508893 20855 heartbeater.cc:503] Master 127.1.74.190:33583 was elected leader, sending a full tablet report...
I20260619 22:25:04.508224 21007 raft_consensus.cc:348] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.509366 21007 raft_consensus.cc:374] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:04.509438 21007 raft_consensus.cc:729] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cedf4b5b0bd249229a9808b1e0ba4e42, State: Initialized, Role: FOLLOWER
I20260619 22:25:04.509819 21007 consensus_queue.cc:260] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.510444 20721 heartbeater.cc:503] Master 127.1.74.190:33583 was elected leader, sending a full tablet report...
I20260619 22:25:04.510614 21007 ts_tablet_manager.cc:1428] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 22:25:04.553362 21013 raft_consensus.cc:482] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:25:04.553634 21013 raft_consensus.cc:504] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.555042 21013 leader_election.cc:288] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cedf4b5b0bd249229a9808b1e0ba4e42 (127.1.74.129:41731), d32f5ff813b6494e93d77511b3edef00 (127.1.74.131:46415)
I20260619 22:25:04.559417 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.555488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:46485 (local address 127.1.74.129:41731)
0619 22:25:04.555629 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.555635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.555650 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:25:04.555711 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.555718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.555776 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.555911 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:04.556242 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.557014 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.558265 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.558736 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.558833 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.558946 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.559193 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.559281 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 22:25:04.559412 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.555215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:41731 (local address 127.1.74.130:46485)
0619 22:25:04.555351 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.555369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.555479 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.555939 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.555942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.555961 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:04.556101 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.556108 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.557137 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.557140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.558128 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.558137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.558899 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.558904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.559029 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.559058 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.559133 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.559224 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 22:25:04.559551 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.555537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:46183 (local address 127.1.74.131:46415)
0619 22:25:04.555691 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.555696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.555859 (+ 163us) server_negotiation.cc:407] Connection header received
0619 22:25:04.556013 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.556016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.556086 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.556137 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:04.556553 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.557324 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.558378 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.558949 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.559092 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.559157 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.559339 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.559413 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:25:04.559765 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.555348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:46415 (local address 127.1.74.130:46183)
0619 22:25:04.555751 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.555763 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:04.555880 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:04.556297 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:04.556300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:04.556316 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:04.556432 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.556438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.557440 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.557443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.558247 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:04.558258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.559077 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.559081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:04.559205 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.559227 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 22:25:04.559305 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 22:25:04.559371 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":5,"thread_start_us":118,"threads_started":1}
I20260619 22:25:04.560204 20675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e692b5e918194fe4b0658f6bc0fba2d2" candidate_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" is_pre_election: true
I20260619 22:25:04.560328 20943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e692b5e918194fe4b0658f6bc0fba2d2" candidate_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d32f5ff813b6494e93d77511b3edef00" is_pre_election: true
I20260619 22:25:04.560576 20943 raft_consensus.cc:2454] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bd7dc603eaa47c7bd0c88606ee4c6f3 in term 0.
I20260619 22:25:04.560576 20675 raft_consensus.cc:2454] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bd7dc603eaa47c7bd0c88606ee4c6f3 in term 0.
I20260619 22:25:04.561151 20743 leader_election.cc:302] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7bd7dc603eaa47c7bd0c88606ee4c6f3, cedf4b5b0bd249229a9808b1e0ba4e42, d32f5ff813b6494e93d77511b3edef00; no voters:
I20260619 22:25:04.561436 21013 raft_consensus.cc:2781] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:25:04.561523 21013 raft_consensus.cc:482] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:25:04.561585 21013 raft_consensus.cc:3037] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:04.562947 21013 raft_consensus.cc:504] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.563388 21013 leader_election.cc:288] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [CANDIDATE]: Term 1 election: Requested vote from peers cedf4b5b0bd249229a9808b1e0ba4e42 (127.1.74.129:41731), d32f5ff813b6494e93d77511b3edef00 (127.1.74.131:46415)
I20260619 22:25:04.563786 20675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e692b5e918194fe4b0658f6bc0fba2d2" candidate_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42"
I20260619 22:25:04.563886 20943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e692b5e918194fe4b0658f6bc0fba2d2" candidate_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d32f5ff813b6494e93d77511b3edef00"
I20260619 22:25:04.563966 20675 raft_consensus.cc:3037] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:04.564029 20943 raft_consensus.cc:3037] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:04.565181 20675 raft_consensus.cc:2454] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bd7dc603eaa47c7bd0c88606ee4c6f3 in term 1.
I20260619 22:25:04.565306 20943 raft_consensus.cc:2454] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bd7dc603eaa47c7bd0c88606ee4c6f3 in term 1.
I20260619 22:25:04.565604 20742 leader_election.cc:302] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bd7dc603eaa47c7bd0c88606ee4c6f3, cedf4b5b0bd249229a9808b1e0ba4e42; no voters:
I20260619 22:25:04.565800 21013 raft_consensus.cc:2781] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:04.566028 21013 raft_consensus.cc:686] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bd7dc603eaa47c7bd0c88606ee4c6f3, State: Running, Role: LEADER
I20260619 22:25:04.566402 21013 consensus_queue.cc:237] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } }
I20260619 22:25:04.569649 20530 catalog_manager.cc:5495] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bd7dc603eaa47c7bd0c88606ee4c6f3 (127.1.74.130). New cstate: current_term: 1 leader_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bd7dc603eaa47c7bd0c88606ee4c6f3" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 33227 } health_report { overall_health: HEALTHY } } }
I20260619 22:25:04.586766 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.581083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.1.74.130:33227)
0619 22:25:04.581239 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.581246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.581266 (+ 20us) server_negotiation.cc:407] Connection header received
0619 22:25:04.581388 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.581395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.581481 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.581617 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:04.581968 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.582930 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.584388 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.584771 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.584888 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.585469 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:04.586090 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:04.586221 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.586486 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.586580 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 22:25:04.594195 20675 raft_consensus.cc:1261] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42 [term 1 FOLLOWER]: Refusing update from remote peer 7bd7dc603eaa47c7bd0c88606ee4c6f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:04.594223 20943 raft_consensus.cc:1261] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 1 FOLLOWER]: Refusing update from remote peer 7bd7dc603eaa47c7bd0c88606ee4c6f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:04.595058 21016 consensus_queue.cc:1035] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d32f5ff813b6494e93d77511b3edef00" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 46415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:04.595305 21013 consensus_queue.cc:1035] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cedf4b5b0bd249229a9808b1e0ba4e42" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 41731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:04.602831 21019 mvcc.cc:205] Tried to move back new op lower bound from 7298694777208815616 to 7298694777106829312. Current Snapshot: MvccSnapshot[applied={T|T < 7298694777208815616}]
I20260619 22:25:04.604671 21021 mvcc.cc:205] Tried to move back new op lower bound from 7298694777208815616 to 7298694777106829312. Current Snapshot: MvccSnapshot[applied={T|T < 7298694777208815616}]
I20260619 22:25:04.611986 20529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:04.612167 20529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:04.614888 20529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 714fb60fb53b4d999374b08a284d5799: Sending DeleteTablet for 3 replicas of tablet e692b5e918194fe4b0658f6bc0fba2d2
I20260619 22:25:04.615864 20655 tablet_service.cc:1514] Processing DeleteTablet for tablet e692b5e918194fe4b0658f6bc0fba2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:04 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:42150
I20260619 22:25:04.615887 20923 tablet_service.cc:1514] Processing DeleteTablet for tablet e692b5e918194fe4b0658f6bc0fba2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:04 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:37324
I20260619 22:25:04.616425 21028 tablet_replica.cc:330] T e692b5e918194fe4b0658f6bc0fba2d2 P cedf4b5b0bd249229a9808b1e0ba4e42: stopping tablet replica
I20260619 22:25:04.616530 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20592
I20260619 22:25:04.616793 20789 tablet_service.cc:1514] Processing DeleteTablet for tablet e692b5e918194fe4b0658f6bc0fba2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:04 UTC) from {username='kudu', principal='kudu/127.1.74.190@KRBTEST.COM'} at 127.0.0.1:47784
I20260619 22:25:04.617328 21029 tablet_replica.cc:330] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: stopping tablet replica
I20260619 22:25:04.617822 21029 raft_consensus.cc:2229] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:25:04.618223 21029 raft_consensus.cc:2258] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:04.620041 21029 ts_tablet_manager.cc:1905] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:25:04.622906 21029 ts_tablet_manager.cc:1918] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:25:04.623016 21029 log.cc:1196] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/e692b5e918194fe4b0658f6bc0fba2d2
I20260619 22:25:04.623353 21029 ts_tablet_manager.cc:1939] T e692b5e918194fe4b0658f6bc0fba2d2 P d32f5ff813b6494e93d77511b3edef00: Deleting consensus metadata
W20260619 22:25:04.624488 20514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:41731 (error 108)
I20260619 22:25:04.624747 20515 catalog_manager.cc:4841] TS d32f5ff813b6494e93d77511b3edef00 (127.1.74.131:46415): tablet e692b5e918194fe4b0658f6bc0fba2d2 (table test-table [id=fdc06ae0ed5341399b0dbda5c0f581b0]) successfully deleted
I20260619 22:25:04.625120 21030 tablet_replica.cc:330] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3: stopping tablet replica
I20260619 22:25:04.625464 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20726
I20260619 22:25:04.625461 21030 raft_consensus.cc:2229] T e692b5e918194fe4b0658f6bc0fba2d2 P 7bd7dc603eaa47c7bd0c88606ee4c6f3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:25:04.625787 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.625505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:41731 (local address 127.0.0.1:42152)
0619 22:25:04.625636 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.625705 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:41731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 22:25:04.627534 20514 catalog_manager.cc:4584] TS cedf4b5b0bd249229a9808b1e0ba4e42 (127.1.74.129:41731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e692b5e918194fe4b0658f6bc0fba2d2: Network error: Client connection negotiation failed: client connection to 127.1.74.129:41731: connect: Connection refused (error 111)
I20260619 22:25:04.631018 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20860
I20260619 22:25:04.632712 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.631220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:33227 (local address 127.0.0.1:47812)
0619 22:25:04.632511 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:04.632592 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:33227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1158}
I20260619 22:25:04.637131 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 20498
2026-06-19T22:25:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2356 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T22:25:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:04Z Disabled control of system clock
I20260619 22:25:04.669013 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39017
--webserver_interface=127.1.74.190
--webserver_port=0
--builtin_ntp_servers=127.1.74.148:37629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 22:25:04.781308 21035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:04.781665 21035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:04.781735 21035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:04.785897 21035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:04.786001 21035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:04.786031 21035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:04.786091 21035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:04.786128 21035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:04.791020 21035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:04.792163 21035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:04.793540 21035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:04.802136 21041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:04.802203 21040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:04.802214 21043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:04.802466 21035 server_base.cc:1034] running on GCE node
I20260619 22:25:04.803164 21035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:04.804273 21035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:04.805464 21035 hybrid_clock.cc:648] HybridClock initialized: now 1781907904805452 us; error 33 us; skew 500 ppm
I20260619 22:25:04.813282 21035 webserver.cc:460] Webserver started at http://127.1.74.190:36485/ using document root <none> and password file <none>
I20260619 22:25:04.813968 21035 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:04.814087 21035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:04.814333 21035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:04.816563 21035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "61e81682b0ed48869ffe26641fa3b9fa"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "206a99e713e15d36939f46355de6000f"
server_key_iv: "6f688903d3ac20dba8af34d6386ccf2e"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.817211 21035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "61e81682b0ed48869ffe26641fa3b9fa"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "206a99e713e15d36939f46355de6000f"
server_key_iv: "6f688903d3ac20dba8af34d6386ccf2e"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.821033 21035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:04.823647 21049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:04.824910 21035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:25:04.825069 21035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "61e81682b0ed48869ffe26641fa3b9fa"
format_stamp: "Formatted at 2026-06-19 22:25:04 on dist-test-slave-k84h"
server_key: "206a99e713e15d36939f46355de6000f"
server_key_iv: "6f688903d3ac20dba8af34d6386ccf2e"
server_key_version: "encryptionkey@0"
I20260619 22:25:04.825193 21035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:04.847751 21035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:04.848536 21035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:04.848752 21035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:04.856081 21035 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39017
I20260619 22:25:04.856088 21101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39017 every 8 connection(s)
I20260619 22:25:04.857048 21035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:04.860227 21102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:04.865687 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21035
I20260619 22:25:04.865828 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:04.865912 21102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: Bootstrap starting.
I20260619 22:25:04.866178 1322 external_mini_cluster.cc:1428] Setting key 0a40b3cd39cb771cb9b56c1f77cc2a25
I20260619 22:25:04.868614 21102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:04.869374 21102 log.cc:824] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:04.871462 21102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: No bootstrap required, opened a new log
I20260619 22:25:04.875818 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:04.867590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.1.74.190:39017)
0619 22:25:04.868223 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:04.868237 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:04.868273 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:25:04.869008 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:04.869026 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:04.869425 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:04.869782 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:04.870242 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.871270 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.872457 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:04.872926 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:04.873051 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:04.873927 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:04.873932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:04.873945 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:04.874339 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:04.874392 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:04.874411 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:04.874495 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:04.874701 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 22:25:04.875050 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":144,"threads_started":1}
I20260619 22:25:04.875730 21102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e81682b0ed48869ffe26641fa3b9fa" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39017 } }
I20260619 22:25:04.875969 21102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:04.876029 21102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e81682b0ed48869ffe26641fa3b9fa, State: Initialized, Role: FOLLOWER
I20260619 22:25:04.876490 21102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e81682b0ed48869ffe26641fa3b9fa" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39017 } }
I20260619 22:25:04.876603 21102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:04.876642 21102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:04.876703 21102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:04.877629 21102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e81682b0ed48869ffe26641fa3b9fa" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39017 } }
I20260619 22:25:04.877969 21102 leader_election.cc:302] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61e81682b0ed48869ffe26641fa3b9fa; no voters:
I20260619 22:25:04.878269 21102 leader_election.cc:288] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:04.878384 21107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:04.878643 21107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [term 1 LEADER]: Becoming Leader. State: Replica: 61e81682b0ed48869ffe26641fa3b9fa, State: Running, Role: LEADER
I20260619 22:25:04.879038 21107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e81682b0ed48869ffe26641fa3b9fa" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39017 } }
I20260619 22:25:04.879652 21102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:04.880918 21109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e81682b0ed48869ffe26641fa3b9fa. Latest consensus state: current_term: 1 leader_uuid: "61e81682b0ed48869ffe26641fa3b9fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e81682b0ed48869ffe26641fa3b9fa" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39017 } } }
I20260619 22:25:04.881057 21109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:04.880895 21108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61e81682b0ed48869ffe26641fa3b9fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61e81682b0ed48869ffe26641fa3b9fa" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39017 } } }
I20260619 22:25:04.881198 21108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa [sys.catalog]: This master's current role is: LEADER
W20260619 22:25:04.884330 21122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:04.884438 21122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:04.884510 21116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:04.885190 21116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:04.890552 21116 catalog_manager.cc:1347] Generated new cluster ID: 130bedc36f9d4513a2686c877dcaf505
I20260619 22:25:04.890632 21116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:04.901594 21116 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:04.902576 21116 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:04.906903 21116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: Generated new TSK 0
I20260619 22:25:04.907737 21116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:04.914412 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.129:0
--local_ip_for_outbound_sockets=127.1.74.129
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39017
--builtin_ntp_servers=127.1.74.148:37629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 22:25:05.032212 21126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:05.032548 21126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:05.032620 21126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:05.036887 21126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:05.037009 21126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:05.037142 21126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.129
I20260619 22:25:05.042379 21126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.74.129
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.129
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:05.043607 21126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:05.045059 21126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:05.055557 21134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:05.055604 21126 server_base.cc:1034] running on GCE node
W20260619 22:25:05.055557 21132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:05.055557 21131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:05.056373 21126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:05.057159 21126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:05.058362 21126 hybrid_clock.cc:648] HybridClock initialized: now 1781907905058324 us; error 58 us; skew 500 ppm
I20260619 22:25:05.060667 21126 webserver.cc:460] Webserver started at http://127.1.74.129:36551/ using document root <none> and password file <none>
I20260619 22:25:05.061322 21126 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:05.061393 21126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:05.061626 21126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:05.063923 21126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data/instance:
uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "9fb9815164e77483c7780b032b31523a"
server_key_iv: "2c306fc2b4117af951be6c2209523260"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.064543 21126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance:
uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "9fb9815164e77483c7780b032b31523a"
server_key_iv: "2c306fc2b4117af951be6c2209523260"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.068629 21126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:05.071415 21140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.072747 21126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 22:25:05.072914 21126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal
uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "9fb9815164e77483c7780b032b31523a"
server_key_iv: "2c306fc2b4117af951be6c2209523260"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.073055 21126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:05.106681 21126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:05.107563 21126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:05.107795 21126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:05.108445 21126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:05.109501 21126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:05.109576 21126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.109651 21126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:05.109712 21126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.119758 21126 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.129:33723
I20260619 22:25:05.119781 21253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.129:33723 every 8 connection(s)
I20260619 22:25:05.120748 21126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/data/info.pb
I20260619 22:25:05.121757 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21126
I20260619 22:25:05.121883 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-0/wal/instance
I20260619 22:25:05.122205 1322 external_mini_cluster.cc:1428] Setting key b593ab7b4ecd5ea9ed522129011b7810
I20260619 22:25:05.124382 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.130:0
--local_ip_for_outbound_sockets=127.1.74.130
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39017
--builtin_ntp_servers=127.1.74.148:37629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 22:25:05.131188 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.122819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.129:39335 (local address 127.1.74.190:39017)
0619 22:25:05.123003 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.123009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.124280 (+ 1271us) server_negotiation.cc:407] Connection header received
0619 22:25:05.125839 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.125843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.125906 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.126019 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:05.127060 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.127608 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.129015 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.129265 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.129386 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.130609 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:05.130613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:05.130616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:05.130639 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:05.130659 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:05.130667 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:05.130750 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.130964 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.131033 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 22:25:05.132112 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.123307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39017 (local address 127.1.74.129:39335)
0619 22:25:05.124133 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.124189 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.125589 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.126223 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.126230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.126326 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:05.126877 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.126920 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.127747 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.127751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.128892 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.128901 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.129436 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.129440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.129612 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.130317 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:25:05.130318 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:05.130364 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:25:05.130436 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:05.130751 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:05.130764 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:05.130772 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.130965 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.131294 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":167,"threads_started":1}
I20260619 22:25:05.133663 21254 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39017
I20260619 22:25:05.133934 21254 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:05.134757 21254 heartbeater.cc:511] Master 127.1.74.190:39017 requested a full tablet report, sending...
I20260619 22:25:05.136821 21066 ts_manager.cc:194] Registered new tserver with Master: 1be2ca24fd5c41c29cf0b24cc7c46c86 (127.1.74.129:33723)
I20260619 22:25:05.138291 21066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.129:39335
W20260619 22:25:05.240998 21257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:05.241323 21257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:05.241397 21257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:05.245638 21257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:05.245746 21257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:05.245854 21257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.130
I20260619 22:25:05.251051 21257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.74.130
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.130
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:05.252238 21257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:05.253772 21257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:05.263451 21263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:05.263391 21262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:05.263425 21265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:05.263937 21257 server_base.cc:1034] running on GCE node
I20260619 22:25:05.264365 21257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:05.265089 21257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:05.266275 21257 hybrid_clock.cc:648] HybridClock initialized: now 1781907905266262 us; error 27 us; skew 500 ppm
I20260619 22:25:05.268280 21257 webserver.cc:460] Webserver started at http://127.1.74.130:43237/ using document root <none> and password file <none>
I20260619 22:25:05.268889 21257 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:05.268985 21257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:05.269227 21257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:05.271462 21257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data/instance:
uuid: "9a62355e6c6345f7a5977c96405559c1"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "f954afa9949a8a3ae9c3b28a63fec666"
server_key_iv: "1e4bad0614a028f6de0e87dfa696e147"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.272024 21257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance:
uuid: "9a62355e6c6345f7a5977c96405559c1"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "f954afa9949a8a3ae9c3b28a63fec666"
server_key_iv: "1e4bad0614a028f6de0e87dfa696e147"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.275774 21257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:25:05.278342 21271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.279711 21257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:25:05.279840 21257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal
uuid: "9a62355e6c6345f7a5977c96405559c1"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "f954afa9949a8a3ae9c3b28a63fec666"
server_key_iv: "1e4bad0614a028f6de0e87dfa696e147"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.279959 21257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:05.307041 21257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:05.307811 21257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:05.308033 21257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:05.308658 21257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:05.309644 21257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:05.309716 21257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.309788 21257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:05.309831 21257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.319105 21257 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.130:39261
I20260619 22:25:05.319123 21384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.130:39261 every 8 connection(s)
I20260619 22:25:05.320078 21257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/data/info.pb
I20260619 22:25:05.321525 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21257
I20260619 22:25:05.321673 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal/instance
I20260619 22:25:05.322016 1322 external_mini_cluster.cc:1428] Setting key d37e8583beb0a010c3e998a049d4ec4c
I20260619 22:25:05.324191 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.74.131:0
--local_ip_for_outbound_sockets=127.1.74.131
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39017
--builtin_ntp_servers=127.1.74.148:37629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 22:25:05.330770 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.321978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:53989 (local address 127.1.74.190:39017)
0619 22:25:05.322131 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.322134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.323488 (+ 1354us) server_negotiation.cc:407] Connection header received
0619 22:25:05.325092 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.325096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.325153 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.325236 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:05.326181 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.326995 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.328342 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.328825 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.328958 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.330161 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:05.330166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:05.330173 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:05.330193 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:05.330221 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:05.330235 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:05.330324 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.330472 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.330547 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:25:05.331651 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.322518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39017 (local address 127.1.74.130:53989)
0619 22:25:05.323312 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.323382 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.324853 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.325422 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.325429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.325518 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:05.326020 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.326032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.327154 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.327157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.328212 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.328223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.328986 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.328990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.329143 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.329878 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:25:05.329880 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:05.329923 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:25:05.329990 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:05.330327 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:05.330334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:05.330338 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.330437 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.330680 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":112,"threads_started":1}
I20260619 22:25:05.332928 21385 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39017
I20260619 22:25:05.333236 21385 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:05.333961 21385 heartbeater.cc:511] Master 127.1.74.190:39017 requested a full tablet report, sending...
I20260619 22:25:05.335405 21066 ts_manager.cc:194] Registered new tserver with Master: 9a62355e6c6345f7a5977c96405559c1 (127.1.74.130:39261)
I20260619 22:25:05.336066 21066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.130:53989
W20260619 22:25:05.441670 21388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:05.441990 21388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:05.442066 21388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:05.446247 21388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:05.446345 21388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:05.446439 21388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.74.131
I20260619 22:25:05.451300 21388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.74.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.74.131
--webserver_port=0
--tserver_master_addrs=127.1.74.190:39017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.74.131
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:05.452491 21388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:05.453868 21388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:05.463513 21396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:05.463518 21393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:05.463629 21394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:05.463891 21388 server_base.cc:1034] running on GCE node
I20260619 22:25:05.464237 21388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:05.464869 21388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:05.466025 21388 hybrid_clock.cc:648] HybridClock initialized: now 1781907905465993 us; error 45 us; skew 500 ppm
I20260619 22:25:05.467824 21388 webserver.cc:460] Webserver started at http://127.1.74.131:42127/ using document root <none> and password file <none>
I20260619 22:25:05.468395 21388 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:05.468457 21388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:05.468674 21388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:05.470757 21388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data/instance:
uuid: "3f9382e6090346ee8d49f806deb4daf5"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "c3160edfb6df98222e9057ba6e9245a4"
server_key_iv: "3e276659ed4835ca2876eeb20dd5807f"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.471374 21388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance:
uuid: "3f9382e6090346ee8d49f806deb4daf5"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "c3160edfb6df98222e9057ba6e9245a4"
server_key_iv: "3e276659ed4835ca2876eeb20dd5807f"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.474821 21388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:05.477360 21402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.478618 21388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:25:05.478757 21388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal
uuid: "3f9382e6090346ee8d49f806deb4daf5"
format_stamp: "Formatted at 2026-06-19 22:25:05 on dist-test-slave-k84h"
server_key: "c3160edfb6df98222e9057ba6e9245a4"
server_key_iv: "3e276659ed4835ca2876eeb20dd5807f"
server_key_version: "encryptionkey@0"
I20260619 22:25:05.478928 21388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:05.495100 21388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:05.495822 21388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:05.496032 21388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:05.496630 21388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 22:25:05.497589 21388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 22:25:05.497637 21388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.497707 21388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 22:25:05.497745 21388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:05.507038 21388 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.131:34429
I20260619 22:25:05.507141 21515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.131:34429 every 8 connection(s)
I20260619 22:25:05.507999 21388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/data/info.pb
I20260619 22:25:05.511283 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21388
I20260619 22:25:05.511379 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal/instance
I20260619 22:25:05.511685 1322 external_mini_cluster.cc:1428] Setting key e93c24f59cf5b20804ba7d9044b86f8e
I20260619 22:25:05.517302 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.509918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.131:37887 (local address 127.1.74.190:39017)
0619 22:25:05.510061 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.510065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.511160 (+ 1095us) server_negotiation.cc:407] Connection header received
0619 22:25:05.512711 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.512713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.512761 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.512873 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:05.513793 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.514338 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.515487 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.515719 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.515802 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.516884 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:05.516887 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:05.516889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:05.516902 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:05.516923 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:05.516931 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:05.516991 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.517125 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.517179 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:25:05.518235 21518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.510315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39017 (local address 127.1.74.131:37887)
0619 22:25:05.510991 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.511032 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.512474 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.513041 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.513048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.513141 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:05.513659 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.513672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.514457 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.514460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.515372 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.515382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.515856 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.515859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.516008 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.516622 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 22:25:05.516624 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:05.516667 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 22:25:05.516738 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:05.517004 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:05.517010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:05.517014 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.517111 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.517346 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260619 22:25:05.519614 21516 heartbeater.cc:347] Connected to a master server at 127.1.74.190:39017
I20260619 22:25:05.519865 21516 heartbeater.cc:464] Registering TS with master...
I20260619 22:25:05.520397 21516 heartbeater.cc:511] Master 127.1.74.190:39017 requested a full tablet report, sending...
I20260619 22:25:05.521486 21066 ts_manager.cc:194] Registered new tserver with Master: 3f9382e6090346ee8d49f806deb4daf5 (127.1.74.131:34429)
I20260619 22:25:05.522092 21066 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.74.131:37887
I20260619 22:25:05.525758 1322 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 22:25:05.531924 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.527823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.1.74.190:39017)
0619 22:25:05.527988 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.527992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.528152 (+ 160us) server_negotiation.cc:407] Connection header received
0619 22:25:05.528319 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.528322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.528363 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.528449 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:05.528742 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.529292 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.530216 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.530432 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.530543 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.531413 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:05.531417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:05.531419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:05.531439 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:05.531459 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:05.531467 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:05.531549 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.531679 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.531737 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:25:05.535184 21066 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 22:25:05.538371 21066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 22:25:05.553146 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.547084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:33723 (local address 127.0.0.1:40762)
0619 22:25:05.547652 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.547713 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.547917 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.548556 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.548563 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.548586 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:05.548744 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.548753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.550195 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.550199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.551169 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.551179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.552532 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.552536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.552706 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.552749 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.552859 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.552983 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":27,"thread_start_us":88,"threads_started":1}
I20260619 22:25:05.553229 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.547264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.1.74.129:33723)
0619 22:25:05.547620 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.547634 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.547826 (+ 192us) server_negotiation.cc:407] Connection header received
0619 22:25:05.548178 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.548185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.548394 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.548564 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:05.548938 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.550079 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.551333 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.552144 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.552254 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.552454 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.552928 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.553039 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":130,"threads_started":1}
I20260619 22:25:05.554167 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.548553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.1.74.130:39261)
0619 22:25:05.549246 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.549262 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.549289 (+ 27us) server_negotiation.cc:407] Connection header received
0619 22:25:05.549368 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.549375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.549575 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.549725 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:05.549981 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.551145 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.552703 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.553512 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.553645 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.553839 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.553894 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.553999 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":114,"threads_started":1}
I20260619 22:25:05.554322 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.547819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:34429 (local address 127.0.0.1:55020)
0619 22:25:05.548086 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.548103 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.548192 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.549578 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.549582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.549601 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:05.549726 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.549734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.551270 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.551277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.552594 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.552603 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.553868 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.553872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.553994 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.554012 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.554110 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.554175 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":99,"thread_start_us":86,"threads_started":1}
I20260619 22:25:05.554713 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.547991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55020 (local address 127.1.74.131:34429)
0619 22:25:05.549132 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.549144 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.549167 (+ 23us) server_negotiation.cc:407] Connection header received
0619 22:25:05.549239 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.549244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.549431 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.549577 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:05.549877 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.551145 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.552732 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.553526 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.553627 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.553803 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.554143 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.554571 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1039,"thread_start_us":163,"threads_started":1}
I20260619 22:25:05.555271 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.548173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:39261 (local address 127.0.0.1:37330)
0619 22:25:05.548869 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.548884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.548968 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.549739 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.549742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.549760 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:05.549850 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.549855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.551324 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.551327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.552561 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.552571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.553644 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.553648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.553739 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.553763 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.554746 (+ 983us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.555135 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":118,"thread_start_us":70,"threads_started":1}
I20260619 22:25:05.556900 21188 tablet_service.cc:1467] Processing CreateTablet for tablet 161b30fbdaf544b0bf96c211f1e2b22a (DEFAULT_TABLE table=test-table [id=df91f24e4d414d4a97ae8c512a58946d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:05.557380 21450 tablet_service.cc:1467] Processing CreateTablet for tablet 161b30fbdaf544b0bf96c211f1e2b22a (DEFAULT_TABLE table=test-table [id=df91f24e4d414d4a97ae8c512a58946d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:05.557845 21188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161b30fbdaf544b0bf96c211f1e2b22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:05.557755 21319 tablet_service.cc:1467] Processing CreateTablet for tablet 161b30fbdaf544b0bf96c211f1e2b22a (DEFAULT_TABLE table=test-table [id=df91f24e4d414d4a97ae8c512a58946d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 22:25:05.558355 21450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161b30fbdaf544b0bf96c211f1e2b22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:05.558661 21319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 161b30fbdaf544b0bf96c211f1e2b22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:05.565060 21533 tablet_bootstrap.cc:492] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: Bootstrap starting.
I20260619 22:25:05.566361 21535 tablet_bootstrap.cc:492] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Bootstrap starting.
I20260619 22:25:05.567190 21534 tablet_bootstrap.cc:492] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Bootstrap starting.
I20260619 22:25:05.567560 21533 tablet_bootstrap.cc:654] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:05.568313 21533 log.cc:824] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:05.568480 21535 tablet_bootstrap.cc:654] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:05.569166 21534 tablet_bootstrap.cc:654] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:05.569264 21535 log.cc:824] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:05.569896 21534 log.cc:824] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:05.570560 21533 tablet_bootstrap.cc:492] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: No bootstrap required, opened a new log
I20260619 22:25:05.570779 21533 ts_tablet_manager.cc:1397] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:25:05.571843 21535 tablet_bootstrap.cc:492] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: No bootstrap required, opened a new log
I20260619 22:25:05.572052 21535 ts_tablet_manager.cc:1397] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 22:25:05.574255 21533 raft_consensus.cc:348] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.574486 21533 raft_consensus.cc:374] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:05.574524 21533 raft_consensus.cc:729] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be2ca24fd5c41c29cf0b24cc7c46c86, State: Initialized, Role: FOLLOWER
I20260619 22:25:05.575064 21533 consensus_queue.cc:260] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.575278 21535 raft_consensus.cc:348] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.575528 21535 raft_consensus.cc:374] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:05.575589 21535 raft_consensus.cc:729] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9382e6090346ee8d49f806deb4daf5, State: Initialized, Role: FOLLOWER
I20260619 22:25:05.575852 21534 tablet_bootstrap.cc:492] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: No bootstrap required, opened a new log
I20260619 22:25:05.575851 21254 heartbeater.cc:503] Master 127.1.74.190:39017 was elected leader, sending a full tablet report...
I20260619 22:25:05.576045 21534 ts_tablet_manager.cc:1397] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260619 22:25:05.576113 21533 ts_tablet_manager.cc:1428] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260619 22:25:05.576104 21535 consensus_queue.cc:260] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.576808 21516 heartbeater.cc:503] Master 127.1.74.190:39017 was elected leader, sending a full tablet report...
I20260619 22:25:05.577143 21535 ts_tablet_manager.cc:1428] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 22:25:05.581676 21534 raft_consensus.cc:348] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.581887 21534 raft_consensus.cc:374] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:05.581969 21534 raft_consensus.cc:729] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a62355e6c6345f7a5977c96405559c1, State: Initialized, Role: FOLLOWER
I20260619 22:25:05.582429 21534 consensus_queue.cc:260] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.583123 21385 heartbeater.cc:503] Master 127.1.74.190:39017 was elected leader, sending a full tablet report...
I20260619 22:25:05.583421 21534 ts_tablet_manager.cc:1428] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260619 22:25:05.622014 21255 tablet.cc:2369] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:25:05.715248 21541 raft_consensus.cc:482] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:25:05.715461 21541 raft_consensus.cc:504] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.716538 21541 leader_election.cc:288] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1be2ca24fd5c41c29cf0b24cc7c46c86 (127.1.74.129:33723), 3f9382e6090346ee8d49f806deb4daf5 (127.1.74.131:34429)
I20260619 22:25:05.721588 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.716994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:46803 (local address 127.1.74.129:33723)
0619 22:25:05.717149 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.717155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.717172 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:25:05.717342 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.717347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.717409 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.717481 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:05.717920 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.718711 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.720345 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.720943 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.721076 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.721119 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.721380 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.721448 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 22:25:05.721590 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.717327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.74.130:54217 (local address 127.1.74.131:34429)
0619 22:25:05.717499 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.717506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.717816 (+ 310us) server_negotiation.cc:407] Connection header received
0619 22:25:05.717924 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.717926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.717977 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.718071 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 22:25:05.718310 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.718995 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.720342 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.720943 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.721071 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.721120 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.721380 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.721449 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 22:25:05.721619 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.716888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:33723 (local address 127.1.74.130:46803)
0619 22:25:05.717052 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.717081 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.717201 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.717603 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.717608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.717630 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:05.717767 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.717777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.718825 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.718827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.720209 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.720220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.721084 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.721088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.721235 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.721272 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.721351 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.721445 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":121}
I20260619 22:25:05.721627 21542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.717196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.131:34429 (local address 127.1.74.130:54217)
0619 22:25:05.717602 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.717615 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:05.717705 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:05.718100 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:05.718102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:05.718116 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 22:25:05.718202 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.718206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.719089 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.719092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.720224 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:05.720230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.721084 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.721088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:05.721231 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.721280 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 22:25:05.721351 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 22:25:05.721445 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":29,"thread_start_us":171,"threads_started":1}
I20260619 22:25:05.722412 21470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161b30fbdaf544b0bf96c211f1e2b22a" candidate_uuid: "9a62355e6c6345f7a5977c96405559c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9382e6090346ee8d49f806deb4daf5" is_pre_election: true
I20260619 22:25:05.722746 21470 raft_consensus.cc:2454] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a62355e6c6345f7a5977c96405559c1 in term 0.
I20260619 22:25:05.722898 21208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161b30fbdaf544b0bf96c211f1e2b22a" candidate_uuid: "9a62355e6c6345f7a5977c96405559c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" is_pre_election: true
I20260619 22:25:05.723167 21208 raft_consensus.cc:2454] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a62355e6c6345f7a5977c96405559c1 in term 0.
I20260619 22:25:05.723337 21273 leader_election.cc:302] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9382e6090346ee8d49f806deb4daf5, 9a62355e6c6345f7a5977c96405559c1; no voters:
I20260619 22:25:05.723641 21541 raft_consensus.cc:2781] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 22:25:05.723744 21541 raft_consensus.cc:482] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 22:25:05.723805 21541 raft_consensus.cc:3037] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:05.724830 21541 raft_consensus.cc:504] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.725203 21541 leader_election.cc:288] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [CANDIDATE]: Term 1 election: Requested vote from peers 1be2ca24fd5c41c29cf0b24cc7c46c86 (127.1.74.129:33723), 3f9382e6090346ee8d49f806deb4daf5 (127.1.74.131:34429)
I20260619 22:25:05.725704 21470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161b30fbdaf544b0bf96c211f1e2b22a" candidate_uuid: "9a62355e6c6345f7a5977c96405559c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9382e6090346ee8d49f806deb4daf5"
I20260619 22:25:05.725704 21208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161b30fbdaf544b0bf96c211f1e2b22a" candidate_uuid: "9a62355e6c6345f7a5977c96405559c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86"
I20260619 22:25:05.725847 21208 raft_consensus.cc:3037] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:05.725859 21470 raft_consensus.cc:3037] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:05.727301 21470 raft_consensus.cc:2454] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a62355e6c6345f7a5977c96405559c1 in term 1.
I20260619 22:25:05.727301 21208 raft_consensus.cc:2454] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a62355e6c6345f7a5977c96405559c1 in term 1.
I20260619 22:25:05.727664 21273 leader_election.cc:302] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1be2ca24fd5c41c29cf0b24cc7c46c86, 9a62355e6c6345f7a5977c96405559c1; no voters:
I20260619 22:25:05.727878 21541 raft_consensus.cc:2781] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:05.728176 21541 raft_consensus.cc:686] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 1 LEADER]: Becoming Leader. State: Replica: 9a62355e6c6345f7a5977c96405559c1, State: Running, Role: LEADER
I20260619 22:25:05.728525 21541 consensus_queue.cc:237] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } }
I20260619 22:25:05.731993 21065 catalog_manager.cc:5495] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a62355e6c6345f7a5977c96405559c1 (127.1.74.130). New cstate: current_term: 1 leader_uuid: "9a62355e6c6345f7a5977c96405559c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a62355e6c6345f7a5977c96405559c1" member_type: VOTER last_known_addr { host: "127.1.74.130" port: 39261 } health_report { overall_health: HEALTHY } } }
I20260619 22:25:05.744614 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.740958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.1.74.130:39261)
0619 22:25:05.741108 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:05.741115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:05.741132 (+ 17us) server_negotiation.cc:407] Connection header received
0619 22:25:05.741225 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:05.741227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:05.741282 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:05.741392 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:05.741654 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.742172 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.743110 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:05.743337 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:05.743446 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:05.743755 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:05.744203 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:05.744341 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:05.744384 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 22:25:05.744438 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 22:25:05.750830 21208 raft_consensus.cc:1261] T 161b30fbdaf544b0bf96c211f1e2b22a P 1be2ca24fd5c41c29cf0b24cc7c46c86 [term 1 FOLLOWER]: Refusing update from remote peer 9a62355e6c6345f7a5977c96405559c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:05.750829 21470 raft_consensus.cc:1261] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 1 FOLLOWER]: Refusing update from remote peer 9a62355e6c6345f7a5977c96405559c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:05.751638 21541 consensus_queue.cc:1035] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1be2ca24fd5c41c29cf0b24cc7c46c86" member_type: VOTER last_known_addr { host: "127.1.74.129" port: 33723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:05.751848 21543 consensus_queue.cc:1035] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9382e6090346ee8d49f806deb4daf5" member_type: VOTER last_known_addr { host: "127.1.74.131" port: 34429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 22:25:05.758952 21517 tablet.cc:2369] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 22:25:05.760149 21548 mvcc.cc:205] Tried to move back new op lower bound from 7298694781947613184 to 7298694781866934272. Current Snapshot: MvccSnapshot[applied={T|T < 7298694781947613184}]
I20260619 22:25:05.771240 21065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:05.771421 21065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 22:25:05.774883 21065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61e81682b0ed48869ffe26641fa3b9fa: Sending DeleteTablet for 3 replicas of tablet 161b30fbdaf544b0bf96c211f1e2b22a
I20260619 22:25:05.776047 21450 tablet_service.cc:1514] Processing DeleteTablet for tablet 161b30fbdaf544b0bf96c211f1e2b22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:05 UTC) from {username='slave'} at 127.0.0.1:55020
I20260619 22:25:05.776046 21188 tablet_service.cc:1514] Processing DeleteTablet for tablet 161b30fbdaf544b0bf96c211f1e2b22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:05 UTC) from {username='slave'} at 127.0.0.1:40762
I20260619 22:25:05.776597 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21126
I20260619 22:25:05.776714 21319 tablet_service.cc:1514] Processing DeleteTablet for tablet 161b30fbdaf544b0bf96c211f1e2b22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 22:25:05 UTC) from {username='slave'} at 127.0.0.1:37330
I20260619 22:25:05.777290 21556 tablet_replica.cc:330] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: stopping tablet replica
I20260619 22:25:05.777690 21556 raft_consensus.cc:2229] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 22:25:05.778234 21556 raft_consensus.cc:2258] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:05.778978 21558 tablet_replica.cc:330] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: stopping tablet replica
I20260619 22:25:05.779379 21558 raft_consensus.cc:2229] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 22:25:05.779937 21558 raft_consensus.cc:2258] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 22:25:05.780146 21556 ts_tablet_manager.cc:1905] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:25:05.781697 21558 ts_tablet_manager.cc:1905] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 22:25:05.783365 21556 ts_tablet_manager.cc:1918] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:25:05.783502 21556 log.cc:1196] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-2/wal/wals/161b30fbdaf544b0bf96c211f1e2b22a
I20260619 22:25:05.783918 21556 ts_tablet_manager.cc:1939] T 161b30fbdaf544b0bf96c211f1e2b22a P 3f9382e6090346ee8d49f806deb4daf5: Deleting consensus metadata
I20260619 22:25:05.785176 21558 ts_tablet_manager.cc:1918] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 22:25:05.785357 21558 log.cc:1196] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Deleting WAL directory at /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781907643212257-1322-0/minicluster-data/ts-1/wal/wals/161b30fbdaf544b0bf96c211f1e2b22a
I20260619 22:25:05.785591 21051 catalog_manager.cc:4841] TS 3f9382e6090346ee8d49f806deb4daf5 (127.1.74.131:34429): tablet 161b30fbdaf544b0bf96c211f1e2b22a (table test-table [id=df91f24e4d414d4a97ae8c512a58946d]) successfully deleted
I20260619 22:25:05.785825 21558 ts_tablet_manager.cc:1939] T 161b30fbdaf544b0bf96c211f1e2b22a P 9a62355e6c6345f7a5977c96405559c1: Deleting consensus metadata
W20260619 22:25:05.786279 21051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.74.129:33723 (error 108)
I20260619 22:25:05.787163 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21257
I20260619 22:25:05.787914 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.787142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.129:33723 (local address 127.0.0.1:40776)
0619 22:25:05.787745 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.787823 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.129:33723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502}
W20260619 22:25:05.789122 21051 catalog_manager.cc:4584] TS 1be2ca24fd5c41c29cf0b24cc7c46c86 (127.1.74.129:33723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 161b30fbdaf544b0bf96c211f1e2b22a: Network error: Client connection negotiation failed: client connection to 127.1.74.129:33723: connect: Connection refused (error 111)
I20260619 22:25:05.792644 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21388
I20260619 22:25:05.793959 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:05.792898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.130:39261 (local address 127.0.0.1:37356)
0619 22:25:05.793770 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:05.793840 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.130:39261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":757}
I20260619 22:25:05.798192 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21035
2026-06-19T22:25:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1160 ms)
[----------] 6 tests from EncryptionPolicyTest (11173 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:05 dist-test-slave-k84h krb5kdc[21562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:05 dist-test-slave-k84h krb5kdc[21562](info): set up 2 sockets
Jun 19 22:25:05 dist-test-slave-k84h krb5kdc[21562](info): commencing operation
krb5kdc: starting...
W20260619 22:25:06.939196 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:07 dist-test-slave-k84h krb5kdc[21562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:07.070811 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:07.184442 21575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:07.184734 21575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:07.184778 21575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:07.189222 21575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:07.189298 21575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:07.189318 21575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:07.189337 21575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:07.189352 21575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:07.194831 21575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:07.195995 21575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:07.197130 21575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:07.206897 21583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:07.206825 21580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:07.206946 21581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:07.207165 21575 server_base.cc:1034] running on GCE node
I20260619 22:25:07.207790 21575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:07.208959 21575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:07.210145 21575 hybrid_clock.cc:648] HybridClock initialized: now 1781907907210132 us; error 27 us; skew 500 ppm
Jun 19 22:25:07 dist-test-slave-k84h krb5kdc[21562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907907, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:07.213231 21575 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:07.214155 21575 webserver.cc:460] Webserver started at http://127.0.0.1:33861/ using document root <none> and password file <none>
I20260619 22:25:07.214704 21575 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:07.214766 21575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:07.215029 21575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:07.217139 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "a34292e4b8d14ff9b40a229157c1379d"
format_stamp: "Formatted at 2026-06-19 22:25:07 on dist-test-slave-k84h"
server_key: "5f9e1608231fac3745ad3b3c3dd45a94"
server_key_iv: "c33272a868a493291f7277f42738393b"
server_key_version: "encryptionkey@0"
I20260619 22:25:07.217665 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "a34292e4b8d14ff9b40a229157c1379d"
format_stamp: "Formatted at 2026-06-19 22:25:07 on dist-test-slave-k84h"
server_key: "5f9e1608231fac3745ad3b3c3dd45a94"
server_key_iv: "c33272a868a493291f7277f42738393b"
server_key_version: "encryptionkey@0"
I20260619 22:25:07.221590 21575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:25:07.224251 21590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:07.225642 21575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:25:07.225759 21575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "a34292e4b8d14ff9b40a229157c1379d"
format_stamp: "Formatted at 2026-06-19 22:25:07 on dist-test-slave-k84h"
server_key: "5f9e1608231fac3745ad3b3c3dd45a94"
server_key_iv: "c33272a868a493291f7277f42738393b"
server_key_version: "encryptionkey@0"
I20260619 22:25:07.225860 21575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:07.245123 21575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:07.248176 21575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:07.248337 21575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:07.255230 21575 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46623
I20260619 22:25:07.255247 21642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46623 every 8 connection(s)
I20260619 22:25:07.256259 21575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:07.258225 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21575
I20260619 22:25:07.258359 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:07.258679 1322 external_mini_cluster.cc:1428] Setting key 75b43c220935861d6f87111617fe70be
I20260619 22:25:07.259449 21643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:07.265141 21643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d: Bootstrap starting.
Jun 19 22:25:07 dist-test-slave-k84h krb5kdc[21562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 22:25:07.267686 21643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:07.268548 21643 log.cc:824] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:07.270541 21643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d: No bootstrap required, opened a new log
I20260619 22:25:07.272594 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:07.260135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35541 (local address 127.0.0.1:46623)
0619 22:25:07.260561 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:07.260578 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:07.260612 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:25:07.261238 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:07.261253 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:07.261569 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:07.261895 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:07.262665 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:07.263733 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:07.264589 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:07.265076 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:07.265241 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:07.267907 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:07.267911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:07.267925 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:07.267961 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:07.269679 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:07.270205 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:07.270210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:07.270214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:07.270323 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:07.270670 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:07.270676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:07.270682 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:07.271132 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:07.271349 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:07.271663 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 22:25:07.271880 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":149,"threads_started":1}
I20260619 22:25:07.274578 21643 raft_consensus.cc:348] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34292e4b8d14ff9b40a229157c1379d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46623 } }
I20260619 22:25:07.274842 21643 raft_consensus.cc:374] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:07.274945 21643 raft_consensus.cc:729] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34292e4b8d14ff9b40a229157c1379d, State: Initialized, Role: FOLLOWER
I20260619 22:25:07.275454 21643 consensus_queue.cc:260] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34292e4b8d14ff9b40a229157c1379d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46623 } }
I20260619 22:25:07.275632 21643 raft_consensus.cc:388] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:07.275692 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:07.275702 21643 raft_consensus.cc:482] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:07.275796 21643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:07.277034 21643 raft_consensus.cc:504] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34292e4b8d14ff9b40a229157c1379d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46623 } }
I20260619 22:25:07.277419 21643 leader_election.cc:302] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a34292e4b8d14ff9b40a229157c1379d; no voters:
I20260619 22:25:07.277768 21643 leader_election.cc:288] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:07.278105 21652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:07.278376 21652 raft_consensus.cc:686] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [term 1 LEADER]: Becoming Leader. State: Replica: a34292e4b8d14ff9b40a229157c1379d, State: Running, Role: LEADER
I20260619 22:25:07.278688 21652 consensus_queue.cc:237] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34292e4b8d14ff9b40a229157c1379d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46623 } }
I20260619 22:25:07.279120 21643 sys_catalog.cc:564] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:07.281083 21654 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a34292e4b8d14ff9b40a229157c1379d. Latest consensus state: current_term: 1 leader_uuid: "a34292e4b8d14ff9b40a229157c1379d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34292e4b8d14ff9b40a229157c1379d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46623 } } }
I20260619 22:25:07.281209 21654 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:07.281476 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a34292e4b8d14ff9b40a229157c1379d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a34292e4b8d14ff9b40a229157c1379d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46623 } } }
I20260619 22:25:07.281731 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d [sys.catalog]: This master's current role is: LEADER
W20260619 22:25:07.284456 21667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:07.284538 21667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:07.284994 21668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:07.287474 21668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:07.288249 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:07.277956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33847 (local address 127.0.0.1:46623)
0619 22:25:07.278070 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:07.278075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:07.279325 (+ 1250us) server_negotiation.cc:407] Connection header received
0619 22:25:07.279478 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:07.279481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:07.279530 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:07.279628 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:07.280507 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:07.281309 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:07.282594 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:07.282970 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:07.283071 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:07.284731 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:07.284733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:07.284735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:07.284756 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:07.286172 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:07.286729 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:07.286733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:07.286735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:07.286829 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:07.287225 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:07.287228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:07.287230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:07.287457 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:07.287630 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:07.287964 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 22:25:07.288071 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 22:25:07.293124 21668 catalog_manager.cc:1347] Generated new cluster ID: a57968fdd0154974967d9adf1ddfcc5d
I20260619 22:25:07.293203 21668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:07.309544 21668 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:07.310616 21668 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:07.317021 21668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a34292e4b8d14ff9b40a229157c1379d: Generated new TSK 0
I20260619 22:25:07.317780 21668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:07.321323 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21575
W20260619 22:25:07.327121 21560 connection.cc:535] client connection to 127.0.0.1:46623 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46623): Connection reset by peer (error 104)
2026-06-19T22:25:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:07 dist-test-slave-k84h krb5kdc[21674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:07 dist-test-slave-k84h krb5kdc[21674](info): set up 2 sockets
Jun 19 22:25:07 dist-test-slave-k84h krb5kdc[21674](info): commencing operation
krb5kdc: starting...
W20260619 22:25:08.367390 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:08 dist-test-slave-k84h krb5kdc[21674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:08.498508 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:08.617748 21687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:08.618127 21687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:08.618211 21687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:08.622426 21687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:08.622566 21687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:08.622605 21687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:08.622643 21687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:08.622680 21687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:08.628310 21687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:08.629498 21687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:08.630607 21687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:08.639667 21695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:08.639690 21693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:08.639751 21687 server_base.cc:1034] running on GCE node
W20260619 22:25:08.639693 21692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:08.640599 21687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:08.641860 21687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:08.643061 21687 hybrid_clock.cc:648] HybridClock initialized: now 1781907908643048 us; error 30 us; skew 500 ppm
Jun 19 22:25:08 dist-test-slave-k84h krb5kdc[21674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907908, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:08.645998 21687 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:08.647083 21687 webserver.cc:460] Webserver started at http://127.0.0.1:44801/ using document root <none> and password file <none>
I20260619 22:25:08.647714 21687 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:08.647780 21687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:08.647959 21687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:08.650064 21687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "43ada397be3941d18b46fcca1a632016"
format_stamp: "Formatted at 2026-06-19 22:25:08 on dist-test-slave-k84h"
server_key: "dc51c88e432728878442ee02aba1ca25"
server_key_iv: "7541f8bf54fd9d36faf4c0ae109e4597"
server_key_version: "encryptionkey@0"
I20260619 22:25:08.650631 21687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "43ada397be3941d18b46fcca1a632016"
format_stamp: "Formatted at 2026-06-19 22:25:08 on dist-test-slave-k84h"
server_key: "dc51c88e432728878442ee02aba1ca25"
server_key_iv: "7541f8bf54fd9d36faf4c0ae109e4597"
server_key_version: "encryptionkey@0"
I20260619 22:25:08.654603 21687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 22:25:08.657239 21702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:08.658623 21687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 22:25:08.658793 21687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "43ada397be3941d18b46fcca1a632016"
format_stamp: "Formatted at 2026-06-19 22:25:08 on dist-test-slave-k84h"
server_key: "dc51c88e432728878442ee02aba1ca25"
server_key_iv: "7541f8bf54fd9d36faf4c0ae109e4597"
server_key_version: "encryptionkey@0"
I20260619 22:25:08.658989 21687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:08.691962 21687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:08.695645 21687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:08.695888 21687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:08.704082 21687 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41109
I20260619 22:25:08.704111 21754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41109 every 8 connection(s)
I20260619 22:25:08.705227 21687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:08.705682 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21687
I20260619 22:25:08.705868 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:08.706204 1322 external_mini_cluster.cc:1428] Setting key f67be2a4690d02adae68c428818be00f
I20260619 22:25:08.708467 21755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:08.714720 21755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016: Bootstrap starting.
I20260619 22:25:08.716245 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:08.707829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49735 (local address 127.0.0.1:41109)
0619 22:25:08.708268 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:08.708297 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:08.708337 (+ 40us) server_negotiation.cc:407] Connection header received
0619 22:25:08.709077 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:08.709095 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:08.709457 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:08.709830 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:08.710275 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:08.711468 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:08.712451 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:08.713005 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:08.713197 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:08.714291 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:08.714296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:08.714310 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:08.714749 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:08.714844 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:08.714913 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:08.715068 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:08.715272 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 22:25:08.715431 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":155,"threads_started":1}
I20260619 22:25:08.717374 21755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:08.718225 21755 log.cc:824] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:08.719625 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:08.720896 21755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016: No bootstrap required, opened a new log
I20260619 22:25:08.725548 21755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ada397be3941d18b46fcca1a632016" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260619 22:25:08.725900 21755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:08.725989 21755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ada397be3941d18b46fcca1a632016, State: Initialized, Role: FOLLOWER
I20260619 22:25:08.726593 21755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ada397be3941d18b46fcca1a632016" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260619 22:25:08.726794 21755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:08.726900 21755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:08.727031 21755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:08.727470 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:08.722167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45993 (local address 127.0.0.1:41109)
0619 22:25:08.722311 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:08.722316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:08.722525 (+ 209us) server_negotiation.cc:407] Connection header received
0619 22:25:08.722609 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:08.722612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:08.722662 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:08.722756 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:08.723061 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:08.724456 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:08.725310 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:08.725681 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:08.725800 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:08.726798 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:08.726804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:08.726811 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:08.726829 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:08.726907 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:08.726929 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:08.727041 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:08.727267 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 22:25:08.727303 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 22:25:08.728441 21755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ada397be3941d18b46fcca1a632016" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260619 22:25:08.728828 21755 leader_election.cc:302] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43ada397be3941d18b46fcca1a632016; no voters:
I20260619 22:25:08.729205 21755 leader_election.cc:288] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:08.729344 21765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:08.729576 21765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [term 1 LEADER]: Becoming Leader. State: Replica: 43ada397be3941d18b46fcca1a632016, State: Running, Role: LEADER
I20260619 22:25:08.729900 21765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ada397be3941d18b46fcca1a632016" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260619 22:25:08.730588 21755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:08.731724 21767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ada397be3941d18b46fcca1a632016. Latest consensus state: current_term: 1 leader_uuid: "43ada397be3941d18b46fcca1a632016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ada397be3941d18b46fcca1a632016" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260619 22:25:08.731865 21767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:08.731698 21766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43ada397be3941d18b46fcca1a632016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ada397be3941d18b46fcca1a632016" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260619 22:25:08.732088 21766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:08.733712 21774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:08.736440 21774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:08.742089 21774 catalog_manager.cc:1347] Generated new cluster ID: a29d8396d2e54a5597a2e6a8bfaa2bfc
I20260619 22:25:08.742184 21774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:08.752871 21774 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:08.753850 21774 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:08.759626 21774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43ada397be3941d18b46fcca1a632016: Generated new TSK 0
I20260619 22:25:08.760325 21774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:08.764917 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21687
W20260619 22:25:08.770572 21672 connection.cc:535] client connection to 127.0.0.1:41109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41109): Connection reset by peer (error 104)
2026-06-19T22:25:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:08 dist-test-slave-k84h krb5kdc[21786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:08 dist-test-slave-k84h krb5kdc[21786](info): set up 2 sockets
Jun 19 22:25:08 dist-test-slave-k84h krb5kdc[21786](info): commencing operation
krb5kdc: starting...
W20260619 22:25:09.803130 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:09 dist-test-slave-k84h krb5kdc[21786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:09.936818 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:10.052048 21799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:10.052415 21799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:10.052459 21799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:10.056722 21799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:10.056802 21799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:10.056823 21799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:10.056839 21799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:10.056855 21799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:10.062465 21799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:10.063652 21799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:10.064672 21799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:10.073702 21805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:10.073766 21799 server_base.cc:1034] running on GCE node
W20260619 22:25:10.073772 21807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:10.073702 21804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:10.074604 21799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:10.075831 21799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:10.077028 21799 hybrid_clock.cc:648] HybridClock initialized: now 1781907910076995 us; error 52 us; skew 500 ppm
Jun 19 22:25:10 dist-test-slave-k84h krb5kdc[21786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:10.085276 21799 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:10.086386 21799 webserver.cc:460] Webserver started at http://127.0.0.1:39181/ using document root <none> and password file <none>
I20260619 22:25:10.087081 21799 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:10.087179 21799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:10.087415 21799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:10.089582 21799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "59242b9c65b24b958d2bb3601c861c14"
format_stamp: "Formatted at 2026-06-19 22:25:10 on dist-test-slave-k84h"
server_key: "be5080154aa15cbfbb7831d3caaaec9f"
server_key_iv: "16d48c88f335fdbafdc954bca6784f7e"
server_key_version: "encryptionkey@0"
I20260619 22:25:10.090170 21799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "59242b9c65b24b958d2bb3601c861c14"
format_stamp: "Formatted at 2026-06-19 22:25:10 on dist-test-slave-k84h"
server_key: "be5080154aa15cbfbb7831d3caaaec9f"
server_key_iv: "16d48c88f335fdbafdc954bca6784f7e"
server_key_version: "encryptionkey@0"
I20260619 22:25:10.094113 21799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260619 22:25:10.096817 21814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:10.098229 21799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:25:10.098419 21799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "59242b9c65b24b958d2bb3601c861c14"
format_stamp: "Formatted at 2026-06-19 22:25:10 on dist-test-slave-k84h"
server_key: "be5080154aa15cbfbb7831d3caaaec9f"
server_key_iv: "16d48c88f335fdbafdc954bca6784f7e"
server_key_version: "encryptionkey@0"
I20260619 22:25:10.098560 21799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:10.103685 21799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:10.106988 21799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:10.107205 21799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:10.114492 21799 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42869
I20260619 22:25:10.114518 21866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42869 every 8 connection(s)
I20260619 22:25:10.115595 21799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:10.118377 21867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:10.124011 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21799
I20260619 22:25:10.124172 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:10.124367 21867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14: Bootstrap starting.
I20260619 22:25:10.124467 1322 external_mini_cluster.cc:1428] Setting key 947aaa3f608b769591521bf9e080c6b5
I20260619 22:25:10.127097 21867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:10.127843 21867 log.cc:824] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:10.130148 21867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14: No bootstrap required, opened a new log
I20260619 22:25:10.130434 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:10.125980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47343 (local address 127.0.0.1:42869)
0619 22:25:10.126506 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:10.126522 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:10.126559 (+ 37us) server_negotiation.cc:407] Connection header received
0619 22:25:10.127405 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:10.127425 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:10.127745 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:10.128106 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:10.129044 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:10.129050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:10.129063 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:10.129075 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:10.129140 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:10.129172 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:10.129278 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:10.129417 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 22:25:10.129653 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":161,"threads_started":1}
I20260619 22:25:10.133703 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:10.134757 21867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59242b9c65b24b958d2bb3601c861c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42869 } }
I20260619 22:25:10.135115 21867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:10.135267 21867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59242b9c65b24b958d2bb3601c861c14, State: Initialized, Role: FOLLOWER
I20260619 22:25:10.135788 21867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59242b9c65b24b958d2bb3601c861c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42869 } }
I20260619 22:25:10.135975 21867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:10.136085 21867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:10.136209 21867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:10.137655 21867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59242b9c65b24b958d2bb3601c861c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42869 } }
I20260619 22:25:10.137797 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:10.136158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49297 (local address 127.0.0.1:42869)
0619 22:25:10.136296 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:10.136300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:10.136450 (+ 150us) server_negotiation.cc:407] Connection header received
0619 22:25:10.136536 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:10.136540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:10.136581 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:10.136680 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:10.137375 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:10.137378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:10.137380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:10.137391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:10.137420 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:10.137435 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:10.137535 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:10.137593 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 22:25:10.137645 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:25:10.138084 21867 leader_election.cc:302] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59242b9c65b24b958d2bb3601c861c14; no voters:
I20260619 22:25:10.138435 21867 leader_election.cc:288] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:10.138557 21877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:10.138797 21877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [term 1 LEADER]: Becoming Leader. State: Replica: 59242b9c65b24b958d2bb3601c861c14, State: Running, Role: LEADER
I20260619 22:25:10.139206 21877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59242b9c65b24b958d2bb3601c861c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42869 } }
I20260619 22:25:10.139726 21867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:10.140887 21878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59242b9c65b24b958d2bb3601c861c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59242b9c65b24b958d2bb3601c861c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42869 } } }
I20260619 22:25:10.140908 21879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59242b9c65b24b958d2bb3601c861c14. Latest consensus state: current_term: 1 leader_uuid: "59242b9c65b24b958d2bb3601c861c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59242b9c65b24b958d2bb3601c861c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42869 } } }
I20260619 22:25:10.141017 21878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:10.141016 21879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:10.141384 21886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:10.144177 21886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:10.149667 21886 catalog_manager.cc:1347] Generated new cluster ID: e8adb42dd3a74ca59dc1d2fe2a451367
I20260619 22:25:10.149736 21886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:10.161352 21886 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:10.161695 21886 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:10.167696 21886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59242b9c65b24b958d2bb3601c861c14: Generated new TSK 0
I20260619 22:25:10.168538 21886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:10.172356 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21799
2026-06-19T22:25:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:10 dist-test-slave-k84h krb5kdc[21898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:10 dist-test-slave-k84h krb5kdc[21898](info): set up 2 sockets
Jun 19 22:25:10 dist-test-slave-k84h krb5kdc[21898](info): commencing operation
krb5kdc: starting...
W20260619 22:25:11.219331 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:11 dist-test-slave-k84h krb5kdc[21898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:11.346979 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42657
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:11.461264 21911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:11.461577 21911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:11.461706 21911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:11.466056 21911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:11.466142 21911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:11.466163 21911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:11.466180 21911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:11.466214 21911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:11.472015 21911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:11.473284 21911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:11.474375 21911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:11.483446 21916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:11.483434 21917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:11.483434 21919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:11.483898 21911 server_base.cc:1034] running on GCE node
I20260619 22:25:11.484570 21911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:11.486320 21911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:11.487515 21911 hybrid_clock.cc:648] HybridClock initialized: now 1781907911487494 us; error 44 us; skew 500 ppm
Jun 19 22:25:11 dist-test-slave-k84h krb5kdc[21898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:11.490557 21911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:11.491614 21911 webserver.cc:460] Webserver started at http://127.0.0.1:40549/ using document root <none> and password file <none>
I20260619 22:25:11.492332 21911 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:11.492440 21911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:11.492806 21911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:11.495216 21911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "d74d9cff9aa742fba970473233f467eb"
format_stamp: "Formatted at 2026-06-19 22:25:11 on dist-test-slave-k84h"
server_key: "b71b940cf68f001ea93e3ddee3f9488c"
server_key_iv: "c3a27b3629a15c69ec46b822ffe45e74"
server_key_version: "encryptionkey@0"
I20260619 22:25:11.495828 21911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "d74d9cff9aa742fba970473233f467eb"
format_stamp: "Formatted at 2026-06-19 22:25:11 on dist-test-slave-k84h"
server_key: "b71b940cf68f001ea93e3ddee3f9488c"
server_key_iv: "c3a27b3629a15c69ec46b822ffe45e74"
server_key_version: "encryptionkey@0"
I20260619 22:25:11.499877 21911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 22:25:11.502553 21926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:11.504135 21911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:11.504315 21911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "d74d9cff9aa742fba970473233f467eb"
format_stamp: "Formatted at 2026-06-19 22:25:11 on dist-test-slave-k84h"
server_key: "b71b940cf68f001ea93e3ddee3f9488c"
server_key_iv: "c3a27b3629a15c69ec46b822ffe45e74"
server_key_version: "encryptionkey@0"
I20260619 22:25:11.504454 21911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:11.535218 21911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:11.538627 21911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:11.538897 21911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:11.546030 21911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42657
I20260619 22:25:11.546057 21978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42657 every 8 connection(s)
I20260619 22:25:11.547013 21911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:11.549698 21979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:11.554397 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 21911
I20260619 22:25:11.554553 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:11.554829 1322 external_mini_cluster.cc:1428] Setting key 9d31be26dca52a34831417f4c9d362a6
I20260619 22:25:11.555589 21979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb: Bootstrap starting.
I20260619 22:25:11.557993 21979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:11.558979 21979 log.cc:824] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:11.561139 21979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb: No bootstrap required, opened a new log
Jun 19 22:25:11 dist-test-slave-k84h krb5kdc[21898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 22:25:11.565817 21979 raft_consensus.cc:348] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74d9cff9aa742fba970473233f467eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260619 22:25:11.566113 21979 raft_consensus.cc:374] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:11.566222 21979 raft_consensus.cc:729] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74d9cff9aa742fba970473233f467eb, State: Initialized, Role: FOLLOWER
I20260619 22:25:11.566685 21979 consensus_queue.cc:260] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74d9cff9aa742fba970473233f467eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260619 22:25:11.566840 21979 raft_consensus.cc:388] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:11.566933 21979 raft_consensus.cc:482] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:11.567070 21979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:11.568013 21979 raft_consensus.cc:504] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74d9cff9aa742fba970473233f467eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260619 22:25:11.568365 21979 leader_election.cc:302] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d74d9cff9aa742fba970473233f467eb; no voters:
I20260619 22:25:11.568869 21979 leader_election.cc:288] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:11.568940 21984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:11.569288 21984 raft_consensus.cc:686] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [term 1 LEADER]: Becoming Leader. State: Replica: d74d9cff9aa742fba970473233f467eb, State: Running, Role: LEADER
I20260619 22:25:11.569667 21984 consensus_queue.cc:237] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74d9cff9aa742fba970473233f467eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260619 22:25:11.570113 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:11.556189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43979 (local address 127.0.0.1:42657)
0619 22:25:11.556623 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:11.556645 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:11.556714 (+ 69us) server_negotiation.cc:407] Connection header received
0619 22:25:11.557358 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:11.557374 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:11.557709 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:11.557979 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:11.558962 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:11.559983 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:11.561099 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:11.561618 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:11.561738 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:11.564673 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:11.564678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:11.564693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:11.564721 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:11.566726 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:11.567334 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:11.567340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:11.567346 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:11.567469 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:11.567770 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:11.567773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:11.567775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:11.568237 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:11.568437 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:11.568737 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 22:25:11.569070 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":141,"threads_started":1}
I20260619 22:25:11.570379 21979 sys_catalog.cc:564] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:11.571547 21986 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d74d9cff9aa742fba970473233f467eb. Latest consensus state: current_term: 1 leader_uuid: "d74d9cff9aa742fba970473233f467eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74d9cff9aa742fba970473233f467eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } } }
I20260619 22:25:11.571703 21986 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:11.571978 21985 sys_catalog.cc:455] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d74d9cff9aa742fba970473233f467eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d74d9cff9aa742fba970473233f467eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } } }
I20260619 22:25:11.572069 21985 sys_catalog.cc:458] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:11.572095 21989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:11.573343 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:11.575654 21989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:11.581964 21989 catalog_manager.cc:1347] Generated new cluster ID: 7cd36b0929dd4014a7876096a989525e
I20260619 22:25:11.582046 21989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:11.586287 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:11.575509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36439 (local address 127.0.0.1:42657)
0619 22:25:11.575615 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:11.575620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:11.577859 (+ 2239us) server_negotiation.cc:407] Connection header received
0619 22:25:11.578018 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:11.578022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:11.578075 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:11.578178 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:11.578899 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:11.579435 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:11.580457 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:11.580887 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:11.581031 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:11.582832 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:11.582836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:11.582878 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:11.582912 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:11.584527 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:11.584989 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:11.584993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:11.584995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:11.585085 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:11.585369 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:11.585374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:11.585376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:11.585575 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:11.585715 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:11.585958 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 22:25:11.586081 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 22:25:11.592624 21989 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:11.593847 21989 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:11.599575 21989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d74d9cff9aa742fba970473233f467eb: Generated new TSK 0
I20260619 22:25:11.600366 21989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:11.604696 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 21911
2026-06-19T22:25:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:11 dist-test-slave-k84h krb5kdc[22010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:11 dist-test-slave-k84h krb5kdc[22010](info): set up 2 sockets
Jun 19 22:25:11 dist-test-slave-k84h krb5kdc[22010](info): commencing operation
krb5kdc: starting...
W20260619 22:25:12.659513 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:12 dist-test-slave-k84h krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:12.798987 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:12.913067 22023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:12.913450 22023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:12.913517 22023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:12.917871 22023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:12.917989 22023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:12.918030 22023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:12.918092 22023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:12.918130 22023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:12.923796 22023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:12.924961 22023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:12.926119 22023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:12.934965 22028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:12.935006 22029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:12.935039 22023 server_base.cc:1034] running on GCE node
W20260619 22:25:12.934985 22031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:12.935747 22023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:12.936887 22023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:12.938079 22023 hybrid_clock.cc:648] HybridClock initialized: now 1781907912938061 us; error 31 us; skew 500 ppm
Jun 19 22:25:12 dist-test-slave-k84h krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907912, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:12.940989 22023 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:12.942164 22023 webserver.cc:460] Webserver started at http://127.0.0.1:45597/ using document root <none> and password file <none>
I20260619 22:25:12.942795 22023 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:12.942921 22023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:12.943168 22023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:12.945904 22023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "358eb2a354f54cc1b73d8a0999040d76"
format_stamp: "Formatted at 2026-06-19 22:25:12 on dist-test-slave-k84h"
server_key: "c6646e552ee63bfd471208894d771073"
server_key_iv: "11491193ef51f95f201e08c9cf2a4cc9"
server_key_version: "encryptionkey@0"
I20260619 22:25:12.946593 22023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "358eb2a354f54cc1b73d8a0999040d76"
format_stamp: "Formatted at 2026-06-19 22:25:12 on dist-test-slave-k84h"
server_key: "c6646e552ee63bfd471208894d771073"
server_key_iv: "11491193ef51f95f201e08c9cf2a4cc9"
server_key_version: "encryptionkey@0"
I20260619 22:25:12.951190 22023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260619 22:25:12.953960 22038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:12.956204 22023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 22:25:12.956442 22023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "358eb2a354f54cc1b73d8a0999040d76"
format_stamp: "Formatted at 2026-06-19 22:25:12 on dist-test-slave-k84h"
server_key: "c6646e552ee63bfd471208894d771073"
server_key_iv: "11491193ef51f95f201e08c9cf2a4cc9"
server_key_version: "encryptionkey@0"
I20260619 22:25:12.956630 22023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:12.974013 22023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:12.978149 22023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:12.978394 22023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:12.985734 22023 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36659
I20260619 22:25:12.985752 22090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36659 every 8 connection(s)
I20260619 22:25:12.986820 22023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:12.989722 22091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:12.995440 22091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76: Bootstrap starting.
I20260619 22:25:12.996702 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22023
I20260619 22:25:12.996872 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:12.997176 1322 external_mini_cluster.cc:1428] Setting key ec4e447f04cc11d76d3822a3675d3a59
I20260619 22:25:12.998415 22091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:12.999279 22091 log.cc:824] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:13.001461 22091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76: No bootstrap required, opened a new log
I20260619 22:25:13.005888 22091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358eb2a354f54cc1b73d8a0999040d76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260619 22:25:13.006237 22091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:13.006319 22091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358eb2a354f54cc1b73d8a0999040d76, State: Initialized, Role: FOLLOWER
I20260619 22:25:13.006770 22091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358eb2a354f54cc1b73d8a0999040d76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260619 22:25:13.006960 22091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:13.007050 22091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:13.007148 22091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:13.007295 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:12.998412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56339 (local address 127.0.0.1:36659)
0619 22:25:12.999035 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:12.999054 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:12.999092 (+ 38us) server_negotiation.cc:407] Connection header received
0619 22:25:12.999887 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:12.999907 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:13.000285 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:13.000608 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:13.000950 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:13.002095 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:13.003370 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:13.003929 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:13.004084 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:13.005090 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:13.005096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:13.005111 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:13.005558 (+ 447us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:13.005622 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:13.005646 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:13.005730 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:13.005906 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 22:25:13.006181 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":191,"threads_started":1}
I20260619 22:25:13.008402 22091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358eb2a354f54cc1b73d8a0999040d76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260619 22:25:13.008781 22091 leader_election.cc:302] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 358eb2a354f54cc1b73d8a0999040d76; no voters:
I20260619 22:25:13.009127 22091 leader_election.cc:288] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:13.009246 22096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:13.009518 22096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [term 1 LEADER]: Becoming Leader. State: Replica: 358eb2a354f54cc1b73d8a0999040d76, State: Running, Role: LEADER
I20260619 22:25:13.009877 22096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358eb2a354f54cc1b73d8a0999040d76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } }
I20260619 22:25:13.010452 22091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:13.010743 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:13.011222 22096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 358eb2a354f54cc1b73d8a0999040d76. Latest consensus state: current_term: 1 leader_uuid: "358eb2a354f54cc1b73d8a0999040d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358eb2a354f54cc1b73d8a0999040d76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } } }
I20260619 22:25:13.011369 22096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:13.012187 22097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "358eb2a354f54cc1b73d8a0999040d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358eb2a354f54cc1b73d8a0999040d76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36659 } } }
I20260619 22:25:13.012315 22097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76 [sys.catalog]: This master's current role is: LEADER
W20260619 22:25:13.015738 22115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:13.015832 22115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:13.015892 22104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:13.016788 22104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:13.017983 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:13.013780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49673 (local address 127.0.0.1:36659)
0619 22:25:13.013892 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:13.013904 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:13.013920 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:25:13.014036 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:13.014044 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:13.014117 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:13.014217 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:13.014533 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:13.015181 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:13.016145 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:13.016402 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:13.016541 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:13.017457 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:13.017462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:13.017465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:13.017480 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:13.017517 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:13.017530 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:13.017612 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:13.017744 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 22:25:13.017805 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 22:25:13.022755 22104 catalog_manager.cc:1347] Generated new cluster ID: 82812bc7d56f480c97db208cd545efc3
I20260619 22:25:13.022831 22104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:13.049103 22104 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:13.050400 22104 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:13.058038 22104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 358eb2a354f54cc1b73d8a0999040d76: Generated new TSK 0
I20260619 22:25:13.058933 22104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:13.066205 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22023
2026-06-19T22:25:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:13 dist-test-slave-k84h krb5kdc[22122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:13 dist-test-slave-k84h krb5kdc[22122](info): set up 2 sockets
Jun 19 22:25:13 dist-test-slave-k84h krb5kdc[22122](info): commencing operation
krb5kdc: starting...
W20260619 22:25:14.123224 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:14 dist-test-slave-k84h krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:14.254451 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:14.365617 22135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:14.365926 22135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:14.365969 22135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:14.370383 22135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:14.370467 22135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:14.370489 22135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:14.370507 22135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:14.370523 22135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:14.376160 22135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:14.377254 22135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:14.378497 22135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:14.387351 22143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:14.387351 22140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:14.387377 22141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:14.387701 22135 server_base.cc:1034] running on GCE node
I20260619 22:25:14.388306 22135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:14.389499 22135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:14.390748 22135 hybrid_clock.cc:648] HybridClock initialized: now 1781907914390715 us; error 50 us; skew 500 ppm
Jun 19 22:25:14 dist-test-slave-k84h krb5kdc[22122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907914, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:14.393730 22135 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:14.394608 22135 webserver.cc:460] Webserver started at http://127.0.0.1:42629/ using document root <none> and password file <none>
I20260619 22:25:14.395210 22135 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:14.395287 22135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:14.395457 22135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:14.397529 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "f104ee939f43427dbde314080dadf4ac"
format_stamp: "Formatted at 2026-06-19 22:25:14 on dist-test-slave-k84h"
server_key: "e7abbbce0d09ae2a634ba18fad2dbfc2"
server_key_iv: "402d14d219eb84c4d0bfa2bd579536f7"
server_key_version: "encryptionkey@0"
I20260619 22:25:14.398042 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "f104ee939f43427dbde314080dadf4ac"
format_stamp: "Formatted at 2026-06-19 22:25:14 on dist-test-slave-k84h"
server_key: "e7abbbce0d09ae2a634ba18fad2dbfc2"
server_key_iv: "402d14d219eb84c4d0bfa2bd579536f7"
server_key_version: "encryptionkey@0"
I20260619 22:25:14.401980 22135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 22:25:14.404635 22150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:14.406051 22135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:25:14.406167 22135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "f104ee939f43427dbde314080dadf4ac"
format_stamp: "Formatted at 2026-06-19 22:25:14 on dist-test-slave-k84h"
server_key: "e7abbbce0d09ae2a634ba18fad2dbfc2"
server_key_iv: "402d14d219eb84c4d0bfa2bd579536f7"
server_key_version: "encryptionkey@0"
I20260619 22:25:14.406291 22135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:14.411145 22135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:14.414630 22135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:14.414798 22135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:14.422228 22135 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43113
I20260619 22:25:14.422260 22202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43113 every 8 connection(s)
I20260619 22:25:14.423333 22135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:14.426232 22203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:14.430996 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22135
I20260619 22:25:14.431162 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:14.431443 1322 external_mini_cluster.cc:1428] Setting key cd8191e42723840049618ba5870795e8
I20260619 22:25:14.432163 22203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac: Bootstrap starting.
I20260619 22:25:14.434638 22203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:14.435647 22203 log.cc:824] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:14.436995 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:14.432807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39967 (local address 127.0.0.1:43113)
0619 22:25:14.433253 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:14.433275 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:14.433316 (+ 41us) server_negotiation.cc:407] Connection header received
0619 22:25:14.433980 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:14.433997 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:14.434320 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:14.434613 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:14.435567 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:14.435572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:14.435609 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:14.435621 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:14.435679 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:14.435699 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:14.435854 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:14.435989 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 22:25:14.436173 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":154,"threads_started":1}
I20260619 22:25:14.437774 22203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac: No bootstrap required, opened a new log
I20260619 22:25:14.439978 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:14.442437 22203 raft_consensus.cc:348] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f104ee939f43427dbde314080dadf4ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43113 } }
I20260619 22:25:14.442766 22203 raft_consensus.cc:374] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:14.442834 22203 raft_consensus.cc:729] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f104ee939f43427dbde314080dadf4ac, State: Initialized, Role: FOLLOWER
I20260619 22:25:14.443411 22203 consensus_queue.cc:260] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f104ee939f43427dbde314080dadf4ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43113 } }
I20260619 22:25:14.443580 22203 raft_consensus.cc:388] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:14.443651 22203 raft_consensus.cc:482] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:14.443758 22203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:14.444280 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:14.442439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50823 (local address 127.0.0.1:43113)
0619 22:25:14.442584 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:14.442589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:14.442782 (+ 193us) server_negotiation.cc:407] Connection header received
0619 22:25:14.442927 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:14.442933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:14.442975 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:14.443058 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:14.443875 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:14.443878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:14.443881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:14.443893 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:14.443921 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:14.443940 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:14.444010 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:14.444116 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 22:25:14.444147 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:25:14.445086 22203 raft_consensus.cc:504] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f104ee939f43427dbde314080dadf4ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43113 } }
I20260619 22:25:14.445439 22203 leader_election.cc:302] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f104ee939f43427dbde314080dadf4ac; no voters:
I20260619 22:25:14.445793 22203 leader_election.cc:288] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:14.445878 22213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:14.446161 22213 raft_consensus.cc:686] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [term 1 LEADER]: Becoming Leader. State: Replica: f104ee939f43427dbde314080dadf4ac, State: Running, Role: LEADER
I20260619 22:25:14.446548 22213 consensus_queue.cc:237] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f104ee939f43427dbde314080dadf4ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43113 } }
I20260619 22:25:14.447122 22203 sys_catalog.cc:564] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:14.448236 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f104ee939f43427dbde314080dadf4ac. Latest consensus state: current_term: 1 leader_uuid: "f104ee939f43427dbde314080dadf4ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f104ee939f43427dbde314080dadf4ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43113 } } }
I20260619 22:25:14.448366 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:14.448668 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f104ee939f43427dbde314080dadf4ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f104ee939f43427dbde314080dadf4ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43113 } } }
I20260619 22:25:14.448761 22222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:14.448823 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:14.452554 22222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:14.458799 22222 catalog_manager.cc:1347] Generated new cluster ID: d435acce27fe462899329d0a75e865cf
I20260619 22:25:14.459010 22222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:14.471920 22222 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:14.472291 22222 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:14.486450 22222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f104ee939f43427dbde314080dadf4ac: Generated new TSK 0
I20260619 22:25:14.487283 22222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:14.492898 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22135
2026-06-19T22:25:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:14 dist-test-slave-k84h krb5kdc[22234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:14 dist-test-slave-k84h krb5kdc[22234](info): set up 2 sockets
Jun 19 22:25:14 dist-test-slave-k84h krb5kdc[22234](info): commencing operation
krb5kdc: starting...
W20260619 22:25:15.535307 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:15 dist-test-slave-k84h krb5kdc[22234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:15.683887 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:15.809482 22247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:15.810088 22247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:15.810218 22247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:15.814888 22247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:15.814967 22247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:15.814989 22247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:15.815008 22247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:15.815025 22247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:15.822325 22247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:15.823663 22247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:15.824786 22247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:15.834345 22255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:15.834345 22253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:15.834376 22252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:15.834676 22247 server_base.cc:1034] running on GCE node
I20260619 22:25:15.835434 22247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:15.836678 22247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:15.837930 22247 hybrid_clock.cc:648] HybridClock initialized: now 1781907915837900 us; error 49 us; skew 500 ppm
Jun 19 22:25:15 dist-test-slave-k84h krb5kdc[22234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907915, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:15.847386 22247 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:15.849043 22247 webserver.cc:460] Webserver started at http://127.0.0.1:33267/ using document root <none> and password file <none>
I20260619 22:25:15.850010 22247 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:15.850144 22247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:15.850463 22247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:15.853591 22247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "ce2758f1ab8b4fa5a874ff8f173acccc"
format_stamp: "Formatted at 2026-06-19 22:25:15 on dist-test-slave-k84h"
server_key: "b865c094600458eacc7896b2595e45e6"
server_key_iv: "3458030d27415275462d1fd72b077ea4"
server_key_version: "encryptionkey@0"
I20260619 22:25:15.854380 22247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "ce2758f1ab8b4fa5a874ff8f173acccc"
format_stamp: "Formatted at 2026-06-19 22:25:15 on dist-test-slave-k84h"
server_key: "b865c094600458eacc7896b2595e45e6"
server_key_iv: "3458030d27415275462d1fd72b077ea4"
server_key_version: "encryptionkey@0"
I20260619 22:25:15.860280 22247 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 22:25:15.864413 22262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:15.866051 22247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260619 22:25:15.866282 22247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "ce2758f1ab8b4fa5a874ff8f173acccc"
format_stamp: "Formatted at 2026-06-19 22:25:15 on dist-test-slave-k84h"
server_key: "b865c094600458eacc7896b2595e45e6"
server_key_iv: "3458030d27415275462d1fd72b077ea4"
server_key_version: "encryptionkey@0"
I20260619 22:25:15.866451 22247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:15.895238 22247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:15.898666 22247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:15.898936 22247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:15.906387 22247 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35843
I20260619 22:25:15.906418 22314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35843 every 8 connection(s)
I20260619 22:25:15.907385 22247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:15.911760 22315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:15.912467 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22247
I20260619 22:25:15.912606 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:15.913091 1322 external_mini_cluster.cc:1428] Setting key 924feabe4a2e72c0e652bc9873746fcc
I20260619 22:25:15.922278 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc: Bootstrap starting.
Jun 19 22:25:15 dist-test-slave-k84h krb5kdc[22234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 22:25:15.925954 22315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:15.926968 22315 log.cc:824] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:15.929371 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc: No bootstrap required, opened a new log
I20260619 22:25:15.931833 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:15.914650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:38705 (local address 127.0.0.1:35843)
0619 22:25:15.915109 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:15.915137 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:15.915171 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:25:15.916045 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:15.916069 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:15.916577 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:15.916985 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:15.917854 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:15.919219 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:15.920593 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:15.921425 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:15.925072 (+ 3647us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:15.925329 (+ 257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:15.925333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:15.925346 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:15.925364 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:15.927609 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:15.928252 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:15.928260 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:15.928267 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:15.928394 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:15.928860 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:15.928864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:15.928866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:15.929473 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:15.929722 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:15.930068 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 22:25:15.930332 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":166,"threads_started":1}
I20260619 22:25:15.934206 22315 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260619 22:25:15.934502 22315 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:15.934573 22315 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2758f1ab8b4fa5a874ff8f173acccc, State: Initialized, Role: FOLLOWER
I20260619 22:25:15.935132 22315 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260619 22:25:15.935328 22315 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:15.935401 22315 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:15.935495 22315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:15.935518 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:15.936468 22315 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260619 22:25:15.936825 22315 leader_election.cc:302] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce2758f1ab8b4fa5a874ff8f173acccc; no voters:
I20260619 22:25:15.937162 22315 leader_election.cc:288] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:15.937675 22325 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } } }
I20260619 22:25:15.937820 22325 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:15.938093 22324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:15.938203 22324 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [term 1 LEADER]: Becoming Leader. State: Replica: ce2758f1ab8b4fa5a874ff8f173acccc, State: Running, Role: LEADER
I20260619 22:25:15.938522 22324 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } }
I20260619 22:25:15.938701 22315 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:15.940272 22325 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce2758f1ab8b4fa5a874ff8f173acccc. Latest consensus state: current_term: 1 leader_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2758f1ab8b4fa5a874ff8f173acccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35843 } } }
I20260619 22:25:15.940372 22325 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc [sys.catalog]: This master's current role is: LEADER
W20260619 22:25:15.943969 22339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:15.944056 22339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:15.944160 22334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:15.945156 22334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:15.950819 22334 catalog_manager.cc:1347] Generated new cluster ID: a8a763afbc144d8c98146c5f11f26ce4
I20260619 22:25:15.950940 22334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:15.953918 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:15.937881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:55207 (local address 127.0.0.1:35843)
0619 22:25:15.937994 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:15.938000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:15.938064 (+ 64us) server_negotiation.cc:407] Connection header received
0619 22:25:15.938408 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:15.938411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:15.938492 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:15.938582 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:15.940267 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:15.941392 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:15.943008 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:15.943306 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:15.943428 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:15.945403 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:15.945407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:15.945410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:15.945435 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:15.951921 (+ 6486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:15.952562 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:15.952565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:15.952567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:15.952645 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:15.952980 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:15.952982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:15.952984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:15.953182 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:15.953289 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:15.953650 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 22:25:15.953751 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 22:25:15.974669 22334 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:15.976425 22334 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:15.985605 22334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce2758f1ab8b4fa5a874ff8f173acccc: Generated new TSK 0
I20260619 22:25:15.986677 22334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:16.000258 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22247
W20260619 22:25:16.005260 22232 connection.cc:535] client connection to 127.0.0.1:35843 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35843): Connection reset by peer (error 104)
2026-06-19T22:25:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:16 dist-test-slave-k84h krb5kdc[22345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:16 dist-test-slave-k84h krb5kdc[22345](info): set up 2 sockets
Jun 19 22:25:16 dist-test-slave-k84h krb5kdc[22345](info): commencing operation
krb5kdc: starting...
W20260619 22:25:17.047124 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:17 dist-test-slave-k84h krb5kdc[22345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:17.173888 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:17.284452 22358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:17.284797 22358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:17.284874 22358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:17.289155 22358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:17.289263 22358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:17.289304 22358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:17.289368 22358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:17.289409 22358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:17.294924 22358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:17.296082 22358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:17.297138 22358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:17.305493 22363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:17.305528 22364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:17.305529 22366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:17.305778 22358 server_base.cc:1034] running on GCE node
I20260619 22:25:17.306416 22358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:17.307680 22358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:17.308892 22358 hybrid_clock.cc:648] HybridClock initialized: now 1781907917308880 us; error 26 us; skew 500 ppm
Jun 19 22:25:17 dist-test-slave-k84h krb5kdc[22345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:17.311657 22358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:17.312538 22358 webserver.cc:460] Webserver started at http://127.0.0.1:39335/ using document root <none> and password file <none>
I20260619 22:25:17.313128 22358 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:17.313217 22358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:17.313449 22358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:17.315553 22358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f"
format_stamp: "Formatted at 2026-06-19 22:25:17 on dist-test-slave-k84h"
server_key: "56f0914573bfab4ce67b37e984932c9e"
server_key_iv: "ee07fafd3c66f020bc49661d79dc52a3"
server_key_version: "encryptionkey@0"
I20260619 22:25:17.316107 22358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f"
format_stamp: "Formatted at 2026-06-19 22:25:17 on dist-test-slave-k84h"
server_key: "56f0914573bfab4ce67b37e984932c9e"
server_key_iv: "ee07fafd3c66f020bc49661d79dc52a3"
server_key_version: "encryptionkey@0"
I20260619 22:25:17.319692 22358 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 22:25:17.322065 22373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:17.323333 22358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:17.323489 22358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f"
format_stamp: "Formatted at 2026-06-19 22:25:17 on dist-test-slave-k84h"
server_key: "56f0914573bfab4ce67b37e984932c9e"
server_key_iv: "ee07fafd3c66f020bc49661d79dc52a3"
server_key_version: "encryptionkey@0"
I20260619 22:25:17.323621 22358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:17.358664 22358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:17.361668 22358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:17.361886 22358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:17.369025 22358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37889
I20260619 22:25:17.369050 22425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37889 every 8 connection(s)
I20260619 22:25:17.370050 22358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:17.372887 22426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:17.378342 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f: Bootstrap starting.
I20260619 22:25:17.380369 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22358
I20260619 22:25:17.380517 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:17.380823 1322 external_mini_cluster.cc:1428] Setting key 7cdabb6f59958166cc511dc3aeb906b4
I20260619 22:25:17.381196 22426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:17.382010 22426 log.cc:824] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:17.384114 22426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f: No bootstrap required, opened a new log
I20260619 22:25:17.387837 22426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37889 } }
I20260619 22:25:17.388104 22426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:17.388203 22426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee1f81adfa445ce9ecd23fc6fdca02f, State: Initialized, Role: FOLLOWER
I20260619 22:25:17.388674 22426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37889 } }
I20260619 22:25:17.388826 22426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:17.388898 22426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:17.389004 22426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:17.390123 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:17.382032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:44457 (local address 127.0.0.1:37889)
0619 22:25:17.382531 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:17.382552 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:17.382611 (+ 59us) server_negotiation.cc:407] Connection header received
0619 22:25:17.383463 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:17.383486 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:17.383851 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:17.384181 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:17.384600 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:17.385521 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:17.386472 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:17.387034 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:17.387208 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:17.388151 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:17.388158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:17.388173 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:17.388636 (+ 463us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:17.388688 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:17.388711 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:17.388851 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:17.389040 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 22:25:17.389178 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":143,"threads_started":1}
I20260619 22:25:17.390146 22426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37889 } }
I20260619 22:25:17.390496 22426 leader_election.cc:302] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ee1f81adfa445ce9ecd23fc6fdca02f; no voters:
I20260619 22:25:17.390825 22426 leader_election.cc:288] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:17.391012 22431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:17.391266 22431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [term 1 LEADER]: Becoming Leader. State: Replica: 9ee1f81adfa445ce9ecd23fc6fdca02f, State: Running, Role: LEADER
I20260619 22:25:17.391564 22431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37889 } }
I20260619 22:25:17.392732 22431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ee1f81adfa445ce9ecd23fc6fdca02f. Latest consensus state: current_term: 1 leader_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37889 } } }
I20260619 22:25:17.392865 22431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:17.392951 22426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:17.393190 22432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee1f81adfa445ce9ecd23fc6fdca02f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37889 } } }
I20260619 22:25:17.393277 22432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:17.393292 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:17.393373 22435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:17.396813 22435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:17.400141 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:17.395621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:52291 (local address 127.0.0.1:37889)
0619 22:25:17.395753 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:17.395758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:17.396113 (+ 355us) server_negotiation.cc:407] Connection header received
0619 22:25:17.396343 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:17.396346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:17.396392 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:17.396481 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:17.396753 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:17.397543 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:17.398381 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:17.398643 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:17.398757 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:17.399684 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:17.399687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:17.399690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:17.399702 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:17.399734 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:17.399748 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:17.399844 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:17.399961 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 22:25:17.399999 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 22:25:17.402757 22435 catalog_manager.cc:1347] Generated new cluster ID: f26f78e14b5e4fffa02d8fa844455f53
I20260619 22:25:17.402832 22435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:17.423027 22435 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:17.424177 22435 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:17.438726 22435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ee1f81adfa445ce9ecd23fc6fdca02f: Generated new TSK 0
I20260619 22:25:17.439527 22435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:17.447767 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22358
W20260619 22:25:17.453006 22343 connection.cc:535] client connection to 127.0.0.1:37889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37889): Connection reset by peer (error 104)
2026-06-19T22:25:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:17 dist-test-slave-k84h krb5kdc[22457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:17 dist-test-slave-k84h krb5kdc[22457](info): set up 2 sockets
Jun 19 22:25:17 dist-test-slave-k84h krb5kdc[22457](info): commencing operation
krb5kdc: starting...
W20260619 22:25:18.491339 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:18 dist-test-slave-k84h krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:18.622416 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:18.733340 22470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:18.733668 22470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:18.733747 22470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:18.737846 22470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:18.737957 22470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:18.738016 22470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:18.738054 22470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:18.738092 22470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:18.743443 22470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:18.744573 22470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:18.745646 22470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:18.754010 22476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:18.754005 22475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:18.754005 22478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:18.754432 22470 server_base.cc:1034] running on GCE node
I20260619 22:25:18.755121 22470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:18.756209 22470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:18.757407 22470 hybrid_clock.cc:648] HybridClock initialized: now 1781907918757387 us; error 33 us; skew 500 ppm
Jun 19 22:25:18 dist-test-slave-k84h krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907918, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:18.760300 22470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:18.761170 22470 webserver.cc:460] Webserver started at http://127.0.0.1:41527/ using document root <none> and password file <none>
I20260619 22:25:18.761746 22470 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:18.761834 22470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:18.762058 22470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:18.764150 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "60296dfb9a86434787cea641f7ec7f2b"
format_stamp: "Formatted at 2026-06-19 22:25:18 on dist-test-slave-k84h"
server_key: "b76541284ebe467a5a0d0d9e6ddb203e"
server_key_iv: "90b800efbbc8a05df4c8807f2d9248c4"
server_key_version: "encryptionkey@0"
I20260619 22:25:18.764707 22470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "60296dfb9a86434787cea641f7ec7f2b"
format_stamp: "Formatted at 2026-06-19 22:25:18 on dist-test-slave-k84h"
server_key: "b76541284ebe467a5a0d0d9e6ddb203e"
server_key_iv: "90b800efbbc8a05df4c8807f2d9248c4"
server_key_version: "encryptionkey@0"
I20260619 22:25:18.768708 22470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:18.771152 22485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:18.772364 22470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 22:25:18.772501 22470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "60296dfb9a86434787cea641f7ec7f2b"
format_stamp: "Formatted at 2026-06-19 22:25:18 on dist-test-slave-k84h"
server_key: "b76541284ebe467a5a0d0d9e6ddb203e"
server_key_iv: "90b800efbbc8a05df4c8807f2d9248c4"
server_key_version: "encryptionkey@0"
I20260619 22:25:18.772615 22470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:18.777273 22470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:18.780575 22470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:18.780789 22470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:18.788163 22470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39429
I20260619 22:25:18.788188 22537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39429 every 8 connection(s)
I20260619 22:25:18.789166 22470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:18.792130 22538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:18.797606 22538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b: Bootstrap starting.
I20260619 22:25:18.799206 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22470
I20260619 22:25:18.799346 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:18.799665 1322 external_mini_cluster.cc:1428] Setting key 9d4f6b0264946c50702727b447f10a14
I20260619 22:25:18.800511 22538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:18.801481 22538 log.cc:824] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:18.803751 22538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b: No bootstrap required, opened a new log
I20260619 22:25:18.804975 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:18.800930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:46965 (local address 127.0.0.1:39429)
0619 22:25:18.801432 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:18.801447 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:18.801492 (+ 45us) server_negotiation.cc:407] Connection header received
0619 22:25:18.802235 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:18.802252 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:18.802541 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:18.802917 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:18.803743 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:18.803748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:18.803759 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:18.803769 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:18.803846 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:18.803875 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:18.803997 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:18.804097 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 22:25:18.804266 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":139,"threads_started":1}
I20260619 22:25:18.808120 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:18.808151 22538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60296dfb9a86434787cea641f7ec7f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260619 22:25:18.808436 22538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:18.808512 22538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60296dfb9a86434787cea641f7ec7f2b, State: Initialized, Role: FOLLOWER
I20260619 22:25:18.808915 22538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60296dfb9a86434787cea641f7ec7f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260619 22:25:18.809031 22538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:18.809116 22538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:18.809219 22538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:18.810256 22538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60296dfb9a86434787cea641f7ec7f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260619 22:25:18.810663 22538 leader_election.cc:302] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60296dfb9a86434787cea641f7ec7f2b; no voters:
I20260619 22:25:18.811090 22538 leader_election.cc:288] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:18.811216 22548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:18.811475 22548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [term 1 LEADER]: Becoming Leader. State: Replica: 60296dfb9a86434787cea641f7ec7f2b, State: Running, Role: LEADER
I20260619 22:25:18.811847 22548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60296dfb9a86434787cea641f7ec7f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } }
I20260619 22:25:18.812316 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:18.810509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:51225 (local address 127.0.0.1:39429)
0619 22:25:18.810635 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:18.810639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:18.810771 (+ 132us) server_negotiation.cc:407] Connection header received
0619 22:25:18.810934 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:18.810937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:18.810968 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:18.811065 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:18.811850 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:18.811854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:18.811860 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:18.811875 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:18.811904 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:18.811917 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:18.812003 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:18.812163 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 22:25:18.812206 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:25:18.812505 22538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:18.813858 22550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60296dfb9a86434787cea641f7ec7f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60296dfb9a86434787cea641f7ec7f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } } }
I20260619 22:25:18.813882 22549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60296dfb9a86434787cea641f7ec7f2b. Latest consensus state: current_term: 1 leader_uuid: "60296dfb9a86434787cea641f7ec7f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60296dfb9a86434787cea641f7ec7f2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39429 } } }
I20260619 22:25:18.813978 22550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:18.813980 22549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:18.814427 22557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:18.817152 22557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:18.822971 22557 catalog_manager.cc:1347] Generated new cluster ID: a2b8d40f92214976808a3e59151536cd
I20260619 22:25:18.823081 22557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:18.838085 22557 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:18.838436 22557 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:18.843889 22557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60296dfb9a86434787cea641f7ec7f2b: Generated new TSK 0
I20260619 22:25:18.844672 22557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:18.852839 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22470
2026-06-19T22:25:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:18 dist-test-slave-k84h krb5kdc[22569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:18 dist-test-slave-k84h krb5kdc[22569](info): set up 2 sockets
Jun 19 22:25:18 dist-test-slave-k84h krb5kdc[22569](info): commencing operation
krb5kdc: starting...
W20260619 22:25:19.895334 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:19 dist-test-slave-k84h krb5kdc[22569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:20.023105 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:20.134290 22582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:20.134689 22582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:20.134790 22582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:20.138995 22582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:20.139117 22582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:20.139163 22582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:20.139227 22582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:20.139271 22582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:20.144760 22582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:20.145908 22582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:20.147028 22582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:20.155927 22588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:20.155937 22587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:20.156009 22582 server_base.cc:1034] running on GCE node
W20260619 22:25:20.155927 22590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:20.156754 22582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:20.157940 22582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:20.159116 22582 hybrid_clock.cc:648] HybridClock initialized: now 1781907920159097 us; error 35 us; skew 500 ppm
Jun 19 22:25:20 dist-test-slave-k84h krb5kdc[22569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907920, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:20.162015 22582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:20.162981 22582 webserver.cc:460] Webserver started at http://127.0.0.1:41127/ using document root <none> and password file <none>
I20260619 22:25:20.163589 22582 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:20.163702 22582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:20.163941 22582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:20.166016 22582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "5b328c1b3f2045c3bc367aeb8f5afc02"
format_stamp: "Formatted at 2026-06-19 22:25:20 on dist-test-slave-k84h"
server_key: "403e623dd3eb7439a1610d5b74861439"
server_key_iv: "47224e9d8a96acd83d862139b159bf88"
server_key_version: "encryptionkey@0"
I20260619 22:25:20.166613 22582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "5b328c1b3f2045c3bc367aeb8f5afc02"
format_stamp: "Formatted at 2026-06-19 22:25:20 on dist-test-slave-k84h"
server_key: "403e623dd3eb7439a1610d5b74861439"
server_key_iv: "47224e9d8a96acd83d862139b159bf88"
server_key_version: "encryptionkey@0"
I20260619 22:25:20.170511 22582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260619 22:25:20.173273 22597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:20.174610 22582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 22:25:20.174810 22582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "5b328c1b3f2045c3bc367aeb8f5afc02"
format_stamp: "Formatted at 2026-06-19 22:25:20 on dist-test-slave-k84h"
server_key: "403e623dd3eb7439a1610d5b74861439"
server_key_iv: "47224e9d8a96acd83d862139b159bf88"
server_key_version: "encryptionkey@0"
I20260619 22:25:20.175011 22582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:20.198127 22582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:20.201459 22582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:20.201694 22582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:20.208976 22582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41729
I20260619 22:25:20.209003 22649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41729 every 8 connection(s)
I20260619 22:25:20.209940 22582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:20.212664 22650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:20.218016 22650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02: Bootstrap starting.
I20260619 22:25:20.220074 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22582
I20260619 22:25:20.220214 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:20.220523 1322 external_mini_cluster.cc:1428] Setting key 6a144817f9c15e138b4b27715eac3e13
I20260619 22:25:20.220773 22650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:20.221727 22650 log.cc:824] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:20.223820 22650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02: No bootstrap required, opened a new log
I20260619 22:25:20.228065 22650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41729 } }
I20260619 22:25:20.228353 22650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:20.228425 22650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b328c1b3f2045c3bc367aeb8f5afc02, State: Initialized, Role: FOLLOWER
I20260619 22:25:20.228942 22650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41729 } }
I20260619 22:25:20.229136 22650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:20.229209 22650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:20.229305 22650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 22:25:20 dist-test-slave-k84h krb5kdc[22569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 22:25:20.230335 22650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41729 } }
I20260619 22:25:20.230671 22650 leader_election.cc:302] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b328c1b3f2045c3bc367aeb8f5afc02; no voters:
I20260619 22:25:20.231070 22650 leader_election.cc:288] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:20.231173 22655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:20.231334 22655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [term 1 LEADER]: Becoming Leader. State: Replica: 5b328c1b3f2045c3bc367aeb8f5afc02, State: Running, Role: LEADER
I20260619 22:25:20.231608 22655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41729 } }
I20260619 22:25:20.232389 22650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:20.232867 22656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41729 } } }
I20260619 22:25:20.232987 22656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:20.233561 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b328c1b3f2045c3bc367aeb8f5afc02. Latest consensus state: current_term: 1 leader_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b328c1b3f2045c3bc367aeb8f5afc02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41729 } } }
I20260619 22:25:20.233894 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:20.234726 22664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:20.236702 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:20.221686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:37305 (local address 127.0.0.1:41729)
0619 22:25:20.222233 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:20.222249 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:20.222289 (+ 40us) server_negotiation.cc:407] Connection header received
0619 22:25:20.223034 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:20.223053 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:20.223391 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:20.223692 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:20.224694 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:20.225745 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:20.227102 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:20.227584 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:20.227719 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:20.230808 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:20.230818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:20.230837 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:20.230896 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:20.233078 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:20.233821 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:20.233828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:20.233834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:20.233979 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:20.234464 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:20.234471 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:20.234475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:20.235055 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:20.235303 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:20.235660 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 22:25:20.235936 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":179,"threads_started":1}
I20260619 22:25:20.237987 22664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:20.239884 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:20.246188 22664 catalog_manager.cc:1347] Generated new cluster ID: 079e79047a5c40c0809ffc96d9b4773b
I20260619 22:25:20.246263 22664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:20.250450 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:20.242034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:42731 (local address 127.0.0.1:41729)
0619 22:25:20.242152 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:20.242157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:20.242333 (+ 176us) server_negotiation.cc:407] Connection header received
0619 22:25:20.242480 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:20.242483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:20.242536 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:20.242627 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:20.243384 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:20.243920 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:20.244726 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:20.244982 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:20.245089 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:20.246693 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:20.246697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:20.246700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:20.246728 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:20.248500 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:20.249034 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:20.249038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:20.249040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:20.249143 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:20.249487 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:20.249490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:20.249492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:20.249726 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:20.249890 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:20.250160 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 22:25:20.250263 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 22:25:20.258318 22664 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:20.259599 22664 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:20.266294 22664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b328c1b3f2045c3bc367aeb8f5afc02: Generated new TSK 0
I20260619 22:25:20.267246 22664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:20.274648 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22582
W20260619 22:25:20.281324 22567 connection.cc:535] client connection to 127.0.0.1:41729 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41729): Connection reset by peer (error 104)
2026-06-19T22:25:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:20 dist-test-slave-k84h krb5kdc[22681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:20 dist-test-slave-k84h krb5kdc[22681](info): set up 2 sockets
Jun 19 22:25:20 dist-test-slave-k84h krb5kdc[22681](info): commencing operation
krb5kdc: starting...
W20260619 22:25:21.335073 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:21 dist-test-slave-k84h krb5kdc[22681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:21.461966 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:21.571822 22694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:21.572081 22694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:21.572122 22694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:21.576098 22694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:21.576157 22694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:21.576177 22694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:21.576195 22694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:21.576210 22694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:21.583153 22694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:21.584399 22694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:21.585379 22694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:21.597329 22699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:21.597347 22700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:21.597801 22702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:21.598001 22694 server_base.cc:1034] running on GCE node
I20260619 22:25:21.598742 22694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:21.600147 22694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:21.601408 22694 hybrid_clock.cc:648] HybridClock initialized: now 1781907921601389 us; error 59 us; skew 500 ppm
Jun 19 22:25:21 dist-test-slave-k84h krb5kdc[22681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907921, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:21.605103 22694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:21.606335 22694 webserver.cc:460] Webserver started at http://127.0.0.1:36669/ using document root <none> and password file <none>
I20260619 22:25:21.607018 22694 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:21.607086 22694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:21.607316 22694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:21.610265 22694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "7b3284cfcb3141f6849a29fd3a6b234d"
format_stamp: "Formatted at 2026-06-19 22:25:21 on dist-test-slave-k84h"
server_key: "11303910e019551ab1d9e2edbbbf80dd"
server_key_iv: "89703ef0e25141e9aaf7c63b549e91ca"
server_key_version: "encryptionkey@0"
I20260619 22:25:21.611033 22694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "7b3284cfcb3141f6849a29fd3a6b234d"
format_stamp: "Formatted at 2026-06-19 22:25:21 on dist-test-slave-k84h"
server_key: "11303910e019551ab1d9e2edbbbf80dd"
server_key_iv: "89703ef0e25141e9aaf7c63b549e91ca"
server_key_version: "encryptionkey@0"
I20260619 22:25:21.615602 22694 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 22:25:21.618100 22709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:21.619413 22694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:21.619557 22694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "7b3284cfcb3141f6849a29fd3a6b234d"
format_stamp: "Formatted at 2026-06-19 22:25:21 on dist-test-slave-k84h"
server_key: "11303910e019551ab1d9e2edbbbf80dd"
server_key_iv: "89703ef0e25141e9aaf7c63b549e91ca"
server_key_version: "encryptionkey@0"
I20260619 22:25:21.619676 22694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:21.628877 22694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:21.632040 22694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:21.632248 22694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:21.639585 22694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38029
I20260619 22:25:21.639611 22761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38029 every 8 connection(s)
I20260619 22:25:21.640522 22694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:21.643235 22762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:21.648363 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22694
I20260619 22:25:21.648512 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:21.648798 1322 external_mini_cluster.cc:1428] Setting key 3b1a133aca337f309bf3c8c79195aaf7
I20260619 22:25:21.648856 22762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d: Bootstrap starting.
I20260619 22:25:21.651340 22762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:21.652047 22762 log.cc:824] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:21.653769 22762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d: No bootstrap required, opened a new log
I20260619 22:25:21.657238 22762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260619 22:25:21.657473 22762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:21.657553 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:21.650140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:50337 (local address 127.0.0.1:38029)
0619 22:25:21.650567 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:21.650581 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:21.650615 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:25:21.651306 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:21.651323 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:21.651698 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:21.652016 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:21.652407 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:21.653345 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:21.654262 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:21.654716 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:21.654968 (+ 252us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:21.655885 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:21.655890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:21.655902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:21.656268 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:21.656324 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:21.656345 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:21.656486 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:21.656663 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 22:25:21.656805 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260619 22:25:21.657552 22762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3284cfcb3141f6849a29fd3a6b234d, State: Initialized, Role: FOLLOWER
I20260619 22:25:21.658083 22762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260619 22:25:21.658238 22762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:21.658300 22762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:21.658390 22762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:21.659386 22762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260619 22:25:21.659768 22762 leader_election.cc:302] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b3284cfcb3141f6849a29fd3a6b234d; no voters:
I20260619 22:25:21.660140 22762 leader_election.cc:288] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:21.660207 22767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:21.660404 22767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [term 1 LEADER]: Becoming Leader. State: Replica: 7b3284cfcb3141f6849a29fd3a6b234d, State: Running, Role: LEADER
I20260619 22:25:21.660583 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:21.660733 22767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } }
I20260619 22:25:21.661389 22762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:21.662371 22768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } } }
I20260619 22:25:21.662546 22768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:21.662812 22768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b3284cfcb3141f6849a29fd3a6b234d. Latest consensus state: current_term: 1 leader_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3284cfcb3141f6849a29fd3a6b234d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38029 } } }
I20260619 22:25:21.662998 22768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:21.663933 22780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:21.667541 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:21.662611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:51917 (local address 127.0.0.1:38029)
0619 22:25:21.662715 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:21.662719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:21.663427 (+ 708us) server_negotiation.cc:407] Connection header received
0619 22:25:21.663581 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:21.663584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:21.663634 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:21.663687 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:21.664244 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:21.664768 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:21.665643 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:21.665984 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:21.666089 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:21.667060 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:21.667064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:21.667067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:21.667079 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:21.667111 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:21.667127 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:21.667220 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:21.667382 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 22:25:21.667417 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 22:25:21.667914 22780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:21.673897 22780 catalog_manager.cc:1347] Generated new cluster ID: 0860e89ba972431c97602e0d33329b8f
I20260619 22:25:21.674021 22780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:21.679872 22780 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:21.680912 22780 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:21.686699 22780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b3284cfcb3141f6849a29fd3a6b234d: Generated new TSK 0
I20260619 22:25:21.687458 22780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:21.692440 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22694
W20260619 22:25:21.697986 22679 connection.cc:535] client connection to 127.0.0.1:38029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38029): Connection reset by peer (error 104)
2026-06-19T22:25:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:21 dist-test-slave-k84h krb5kdc[22793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:21 dist-test-slave-k84h krb5kdc[22793](info): set up 2 sockets
Jun 19 22:25:21 dist-test-slave-k84h krb5kdc[22793](info): commencing operation
krb5kdc: starting...
W20260619 22:25:22.751307 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:22 dist-test-slave-k84h krb5kdc[22793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 22:25:22.880533 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:22.994064 22806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:22.994351 22806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:22.994401 22806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:22.999377 22806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:22.999457 22806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:22.999478 22806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:22.999495 22806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:22.999511 22806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:23.005419 22806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39015
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:23.006743 22806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:23.007853 22806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:23.016633 22811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:23.016633 22812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:23.016636 22814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:23.016781 22806 server_base.cc:1034] running on GCE node
I20260619 22:25:23.017481 22806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:23.018612 22806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:23.019780 22806 hybrid_clock.cc:648] HybridClock initialized: now 1781907923019780 us; error 55 us; skew 500 ppm
Jun 19 22:25:23 dist-test-slave-k84h krb5kdc[22793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907923, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:23.022733 22806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 22:25:23.023754 22806 webserver.cc:460] Webserver started at http://127.0.0.1:42613/ using document root <none> and password file <none>
I20260619 22:25:23.024349 22806 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:23.024443 22806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:23.024678 22806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:23.026808 22806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e"
format_stamp: "Formatted at 2026-06-19 22:25:23 on dist-test-slave-k84h"
server_key: "cb4b41d61b92daaeb130341bdce987f0"
server_key_iv: "27d0c200c6a593dcfeaabb24c4352869"
server_key_version: "encryptionkey@0"
I20260619 22:25:23.027456 22806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e"
format_stamp: "Formatted at 2026-06-19 22:25:23 on dist-test-slave-k84h"
server_key: "cb4b41d61b92daaeb130341bdce987f0"
server_key_iv: "27d0c200c6a593dcfeaabb24c4352869"
server_key_version: "encryptionkey@0"
I20260619 22:25:23.031226 22806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:23.033672 22821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:23.035022 22806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:23.035161 22806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e"
format_stamp: "Formatted at 2026-06-19 22:25:23 on dist-test-slave-k84h"
server_key: "cb4b41d61b92daaeb130341bdce987f0"
server_key_iv: "27d0c200c6a593dcfeaabb24c4352869"
server_key_version: "encryptionkey@0"
I20260619 22:25:23.035283 22806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:23.040040 22806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:23.043440 22806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:23.043645 22806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:23.050712 22806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39015
I20260619 22:25:23.050732 22873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39015 every 8 connection(s)
I20260619 22:25:23.051712 22806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:23.054585 22874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:23.056963 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22806
I20260619 22:25:23.057132 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:23.057437 1322 external_mini_cluster.cc:1428] Setting key e1616bfc31b8f0849b1a1e31f6c3adda
I20260619 22:25:23.061367 22874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e: Bootstrap starting.
I20260619 22:25:23.063189 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:23.059076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:40903 (local address 127.0.0.1:39015)
0619 22:25:23.059558 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:23.059574 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:23.059610 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:25:23.060406 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:23.060425 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:23.060721 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:23.061107 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:23.061928 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:23.061932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:23.061944 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:23.061954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:23.062004 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:23.062023 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:23.062173 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:23.062256 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 22:25:23.062418 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":146,"threads_started":1}
I20260619 22:25:23.063884 22874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:23.064623 22874 log.cc:824] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:23.066012 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:23.066897 22874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e: No bootstrap required, opened a new log
I20260619 22:25:23.069972 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:23.068216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:35453 (local address 127.0.0.1:39015)
0619 22:25:23.068361 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:23.068366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:23.068534 (+ 168us) server_negotiation.cc:407] Connection header received
0619 22:25:23.068675 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:23.068677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:23.068707 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:23.068780 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:23.069508 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:23.069510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:23.069512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:23.069523 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:23.069549 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:23.069560 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:23.069631 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:23.069736 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 22:25:23.069804 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 22:25:23.071594 22874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260619 22:25:23.071888 22874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:23.071949 22874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6bad8e0c2e4bfba4375ee601a4cf7e, State: Initialized, Role: FOLLOWER
I20260619 22:25:23.072427 22874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260619 22:25:23.072662 22884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 22:25:23.072831 22884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260619 22:25:23.072634 22874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:23.073273 22874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:23.073223 22884 leader_election.cc:302] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b6bad8e0c2e4bfba4375ee601a4cf7e; no voters:
I20260619 22:25:23.073338 22874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:23.073616 22884 leader_election.cc:288] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260619 22:25:23.074440 22874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260619 22:25:23.074613 22874 leader_election.cc:302] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b6bad8e0c2e4bfba4375ee601a4cf7e; no voters:
I20260619 22:25:23.074730 22874 leader_election.cc:288] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:23.074772 22885 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260619 22:25:23.075133 22884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:23.075146 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } } }
I20260619 22:25:23.075309 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:23.075346 22884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [term 1 LEADER]: Becoming Leader. State: Replica: 1b6bad8e0c2e4bfba4375ee601a4cf7e, State: Running, Role: LEADER
I20260619 22:25:23.075657 22884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } }
I20260619 22:25:23.076042 22874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:23.077699 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b6bad8e0c2e4bfba4375ee601a4cf7e. Latest consensus state: current_term: 1 leader_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6bad8e0c2e4bfba4375ee601a4cf7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39015 } } }
I20260619 22:25:23.077818 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:23.078830 22898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:23.081578 22898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:23.087530 22898 catalog_manager.cc:1347] Generated new cluster ID: c60982b667a943cdb7ca825cff7f7929
I20260619 22:25:23.087617 22898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:23.100008 22898 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:23.100421 22898 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:23.108675 22898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b6bad8e0c2e4bfba4375ee601a4cf7e: Generated new TSK 0
I20260619 22:25:23.109566 22898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:23.119359 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22806
2026-06-19T22:25:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:23 dist-test-slave-k84h krb5kdc[22906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:23 dist-test-slave-k84h krb5kdc[22906](info): set up 2 sockets
Jun 19 22:25:23 dist-test-slave-k84h krb5kdc[22906](info): commencing operation
krb5kdc: starting...
W20260619 22:25:24.167286 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:24 dist-test-slave-k84h krb5kdc[22906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:24.296267 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39157
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:24.406404 22919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:24.406827 22919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:24.406945 22919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:24.411149 22919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:24.411227 22919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:24.411247 22919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:24.411265 22919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:24.411281 22919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:24.416831 22919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39157
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:24.417989 22919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:24.419180 22919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:24.428031 22924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:24.428118 22919 server_base.cc:1034] running on GCE node
W20260619 22:25:24.428053 22927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:24.428054 22925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:24.428886 22919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:24.430092 22919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:24.431262 22919 hybrid_clock.cc:648] HybridClock initialized: now 1781907924431234 us; error 45 us; skew 500 ppm
Jun 19 22:25:24 dist-test-slave-k84h krb5kdc[22906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907924, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:24.434152 22919 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:24.435268 22919 webserver.cc:460] Webserver started at http://127.1.74.190:43233/ using document root <none> and password file <none>
I20260619 22:25:24.435856 22919 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:24.435920 22919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:24.436132 22919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:24.438180 22919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "b6ed415a9207471dabfe1231c0b4afd2"
format_stamp: "Formatted at 2026-06-19 22:25:24 on dist-test-slave-k84h"
server_key: "b1615f3b54b6dadedb4ae4c3e095823c"
server_key_iv: "dc118d4ee34902bab4806bd9dbac0a7a"
server_key_version: "encryptionkey@0"
I20260619 22:25:24.438752 22919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "b6ed415a9207471dabfe1231c0b4afd2"
format_stamp: "Formatted at 2026-06-19 22:25:24 on dist-test-slave-k84h"
server_key: "b1615f3b54b6dadedb4ae4c3e095823c"
server_key_iv: "dc118d4ee34902bab4806bd9dbac0a7a"
server_key_version: "encryptionkey@0"
I20260619 22:25:24.442574 22919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:24.445076 22934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:24.446466 22919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 22:25:24.446605 22919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "b6ed415a9207471dabfe1231c0b4afd2"
format_stamp: "Formatted at 2026-06-19 22:25:24 on dist-test-slave-k84h"
server_key: "b1615f3b54b6dadedb4ae4c3e095823c"
server_key_iv: "dc118d4ee34902bab4806bd9dbac0a7a"
server_key_version: "encryptionkey@0"
I20260619 22:25:24.446732 22919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:24.470304 22919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:24.473785 22919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:24.474030 22919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:24.481801 22919 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39157
I20260619 22:25:24.481820 22986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39157 every 8 connection(s)
I20260619 22:25:24.482825 22919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:24.483130 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 22919
I20260619 22:25:24.483315 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:24.483629 1322 external_mini_cluster.cc:1428] Setting key 9b4b75117e9cf0f4f160cee9cabfa816
I20260619 22:25:24.485951 22987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:24.491542 22987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2: Bootstrap starting.
Jun 19 22:25:24 dist-test-slave-k84h krb5kdc[22906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:24.493884 22987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:24.494801 22987 log.cc:824] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:24.496903 22987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2: No bootstrap required, opened a new log
I20260619 22:25:24.499073 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:24.485103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51981 (local address 127.1.74.190:39157)
0619 22:25:24.485539 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:24.485554 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:24.485590 (+ 36us) server_negotiation.cc:407] Connection header received
0619 22:25:24.486342 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:24.486359 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:24.486753 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:24.487156 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:24.488052 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:24.489068 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:24.489970 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:24.490459 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:24.490638 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:24.493597 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:24.493602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:24.493615 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:24.493645 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:24.495846 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:24.496432 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:24.496437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:24.496442 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:24.496560 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:24.496955 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:24.496964 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:24.496966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:24.497419 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:24.497659 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:24.498012 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 22:25:24.498254 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260619 22:25:24.501144 22987 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ed415a9207471dabfe1231c0b4afd2" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39157 } }
I20260619 22:25:24.501467 22987 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:24.501554 22987 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ed415a9207471dabfe1231c0b4afd2, State: Initialized, Role: FOLLOWER
I20260619 22:25:24.502115 22987 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ed415a9207471dabfe1231c0b4afd2" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39157 } }
I20260619 22:25:24.502300 22987 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:24.502388 22987 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:24.502496 22987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:24.502614 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:24.503635 22987 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ed415a9207471dabfe1231c0b4afd2" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39157 } }
I20260619 22:25:24.504060 22987 leader_election.cc:302] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6ed415a9207471dabfe1231c0b4afd2; no voters:
I20260619 22:25:24.504472 22987 leader_election.cc:288] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:24.504549 22996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:24.504765 22996 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [term 1 LEADER]: Becoming Leader. State: Replica: b6ed415a9207471dabfe1231c0b4afd2, State: Running, Role: LEADER
I20260619 22:25:24.505102 22996 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ed415a9207471dabfe1231c0b4afd2" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39157 } }
I20260619 22:25:24.505877 22987 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:24.506570 22996 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6ed415a9207471dabfe1231c0b4afd2. Latest consensus state: current_term: 1 leader_uuid: "b6ed415a9207471dabfe1231c0b4afd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ed415a9207471dabfe1231c0b4afd2" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39157 } } }
I20260619 22:25:24.506783 22996 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:24.507236 23006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:24.507261 22997 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6ed415a9207471dabfe1231c0b4afd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ed415a9207471dabfe1231c0b4afd2" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39157 } } }
I20260619 22:25:24.507443 22997 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:24.511121 23006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:24.515323 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:24.505264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36119 (local address 127.1.74.190:39157)
0619 22:25:24.505406 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:24.505411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:24.505473 (+ 62us) server_negotiation.cc:407] Connection header received
0619 22:25:24.505651 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:24.505654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:24.505707 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:24.505828 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:24.506718 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:24.507594 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:24.508817 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:24.509166 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:24.509265 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:24.511394 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:24.511399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:24.511403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:24.511434 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:24.513151 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:24.513782 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:24.513787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:24.513789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:24.513887 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:24.514288 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:24.514291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:24.514293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:24.514500 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:24.514666 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:24.515017 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 22:25:24.515128 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:25:24.518253 23006 catalog_manager.cc:1347] Generated new cluster ID: ad8dfdb6313d41da94abd12cf2ad7b1d
I20260619 22:25:24.518337 23006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:24.531291 23006 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:24.532279 23006 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:24.539891 23006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6ed415a9207471dabfe1231c0b4afd2: Generated new TSK 0
I20260619 22:25:24.540721 23006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:24.554562 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 22919
W20260619 22:25:24.560078 22904 connection.cc:535] client connection to 127.1.74.190:39157 recv error: Network error: failed to read from TLS socket (remote: 127.1.74.190:39157): Connection reset by peer (error 104)
2026-06-19T22:25:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:24 dist-test-slave-k84h krb5kdc[23018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:24 dist-test-slave-k84h krb5kdc[23018](info): set up 2 sockets
Jun 19 22:25:24 dist-test-slave-k84h krb5kdc[23018](info): commencing operation
krb5kdc: starting...
W20260619 22:25:25.599267 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:25 dist-test-slave-k84h krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:25.726519 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:45837
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:25.837497 23031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:25.837759 23031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:25.837802 23031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:25.841729 23031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:25.841787 23031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:25.841807 23031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:25.841823 23031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:25.841840 23031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:25.847018 23031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:45837
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:45837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:25.847994 23031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:25.848935 23031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:25.857076 23036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:25.857082 23039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:25.857090 23037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:25.857462 23031 server_base.cc:1034] running on GCE node
I20260619 22:25:25.858019 23031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:25.859169 23031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:25.860352 23031 hybrid_clock.cc:648] HybridClock initialized: now 1781907925860326 us; error 41 us; skew 500 ppm
Jun 19 22:25:25 dist-test-slave-k84h krb5kdc[23018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907925, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:25.863278 23031 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:25.864084 23031 webserver.cc:460] Webserver started at http://127.1.74.190:44659/ using document root <none> and password file <none>
I20260619 22:25:25.864643 23031 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:25.864771 23031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:25.864984 23031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:25.866971 23031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "139d6e7fee50426ba4befb000b9f6c4e"
format_stamp: "Formatted at 2026-06-19 22:25:25 on dist-test-slave-k84h"
server_key: "348fc250e6b5d1185a3b0bb675c21786"
server_key_iv: "e131a9a98345fbedf00c36eb158012ad"
server_key_version: "encryptionkey@0"
I20260619 22:25:25.867504 23031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "139d6e7fee50426ba4befb000b9f6c4e"
format_stamp: "Formatted at 2026-06-19 22:25:25 on dist-test-slave-k84h"
server_key: "348fc250e6b5d1185a3b0bb675c21786"
server_key_iv: "e131a9a98345fbedf00c36eb158012ad"
server_key_version: "encryptionkey@0"
I20260619 22:25:25.870996 23031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:25.873353 23046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:25.874598 23031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:25:25.874782 23031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "139d6e7fee50426ba4befb000b9f6c4e"
format_stamp: "Formatted at 2026-06-19 22:25:25 on dist-test-slave-k84h"
server_key: "348fc250e6b5d1185a3b0bb675c21786"
server_key_iv: "e131a9a98345fbedf00c36eb158012ad"
server_key_version: "encryptionkey@0"
I20260619 22:25:25.874974 23031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:25.901113 23031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:25.904245 23031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:25.904456 23031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:25.911931 23031 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:45837
I20260619 22:25:25.911950 23098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:45837 every 8 connection(s)
I20260619 22:25:25.912875 23031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:25.913188 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23031
I20260619 22:25:25.913352 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:25.913655 1322 external_mini_cluster.cc:1428] Setting key 1ea5e87acc9ffb327011219c5fe83dac
I20260619 22:25:25.916213 23099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:25.923336 23099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e: Bootstrap starting.
I20260619 22:25:25.924005 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:25.915092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44481 (local address 127.1.74.190:45837)
0619 22:25:25.915520 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:25.915539 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:25.915582 (+ 43us) server_negotiation.cc:407] Connection header received
0619 22:25:25.916429 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:25.916451 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:25.916866 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:25.917221 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:25.917633 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:25.919195 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:25.920110 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:25.920800 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:25.920960 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:25.921950 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:25.921957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:25.921976 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:25.922489 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:25.922557 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:25.922586 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:25.922715 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:25.922944 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 22:25:25.923079 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":116,"threads_started":1}
I20260619 22:25:25.925947 23099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:25.926784 23099 log.cc:824] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:25.927116 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:25.928779 23099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e: No bootstrap required, opened a new log
I20260619 22:25:25.933315 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:25.929144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43151 (local address 127.1.74.190:45837)
0619 22:25:25.929264 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:25.929268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:25.929409 (+ 141us) server_negotiation.cc:407] Connection header received
0619 22:25:25.929504 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:25.929507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:25.929552 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:25.929629 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:25.929870 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:25.930641 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:25.931584 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:25.931928 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:25.932038 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:25.932901 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:25.932905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:25.932909 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:25.932921 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:25.932948 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:25.932961 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:25.933055 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:25.933152 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 22:25:25.933186 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 22:25:25.933456 23099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6e7fee50426ba4befb000b9f6c4e" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 45837 } }
I20260619 22:25:25.933717 23099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:25.933766 23099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 139d6e7fee50426ba4befb000b9f6c4e, State: Initialized, Role: FOLLOWER
I20260619 22:25:25.934228 23099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6e7fee50426ba4befb000b9f6c4e" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 45837 } }
I20260619 22:25:25.934406 23099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:25.934481 23099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:25.934571 23099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:25.935595 23099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6e7fee50426ba4befb000b9f6c4e" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 45837 } }
I20260619 22:25:25.935892 23099 leader_election.cc:302] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 139d6e7fee50426ba4befb000b9f6c4e; no voters:
I20260619 22:25:25.936139 23099 leader_election.cc:288] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:25.936278 23109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:25.936506 23109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [term 1 LEADER]: Becoming Leader. State: Replica: 139d6e7fee50426ba4befb000b9f6c4e, State: Running, Role: LEADER
I20260619 22:25:25.936843 23109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6e7fee50426ba4befb000b9f6c4e" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 45837 } }
I20260619 22:25:25.937511 23099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:25.938591 23110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "139d6e7fee50426ba4befb000b9f6c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6e7fee50426ba4befb000b9f6c4e" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 45837 } } }
I20260619 22:25:25.938723 23110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:25.938963 23111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 139d6e7fee50426ba4befb000b9f6c4e. Latest consensus state: current_term: 1 leader_uuid: "139d6e7fee50426ba4befb000b9f6c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "139d6e7fee50426ba4befb000b9f6c4e" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 45837 } } }
I20260619 22:25:25.939052 23111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e [sys.catalog]: This master's current role is: LEADER
W20260619 22:25:25.942149 23124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:25.942245 23124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:25.942304 23118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:25.943106 23118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:25.948200 23118 catalog_manager.cc:1347] Generated new cluster ID: dc1d7368a22c41a380428f8b33731776
I20260619 22:25:25.948287 23118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:25.959688 23118 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:25.960664 23118 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:25.967291 23118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 139d6e7fee50426ba4befb000b9f6c4e: Generated new TSK 0
I20260619 22:25:25.968113 23118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:25.981616 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23031
W20260619 22:25:25.988703 23016 connection.cc:535] client connection to 127.1.74.190:45837 recv error: Network error: failed to read from TLS socket (remote: 127.1.74.190:45837): Connection reset by peer (error 104)
2026-06-19T22:25:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:26 dist-test-slave-k84h krb5kdc[23130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:26 dist-test-slave-k84h krb5kdc[23130](info): set up 2 sockets
Jun 19 22:25:26 dist-test-slave-k84h krb5kdc[23130](info): commencing operation
krb5kdc: starting...
W20260619 22:25:27.031450 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:27 dist-test-slave-k84h krb5kdc[23130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:27.159251 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:36211
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:36211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:27.272696 23143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:27.273003 23143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:27.273048 23143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:27.277336 23143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:27.277416 23143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:27.277438 23143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:27.277455 23143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:27.277499 23143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:27.283147 23143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:36211
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:36211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:27.284487 23143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:27.285954 23143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:27.295228 23151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:27.295183 23149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:27.295186 23148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:27.295243 23143 server_base.cc:1034] running on GCE node
I20260619 22:25:27.295949 23143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:27.297080 23143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:27.298249 23143 hybrid_clock.cc:648] HybridClock initialized: now 1781907927298217 us; error 50 us; skew 500 ppm
Jun 19 22:25:27 dist-test-slave-k84h krb5kdc[23130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907927, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:27.305101 23143 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:27.306125 23143 webserver.cc:460] Webserver started at http://127.1.74.190:42381/ using document root <none> and password file <none>
I20260619 22:25:27.306718 23143 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:27.306782 23143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:27.307034 23143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:27.309089 23143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "cd0ef362870c4f59991bc9edc71e1b73"
format_stamp: "Formatted at 2026-06-19 22:25:27 on dist-test-slave-k84h"
server_key: "8ec8ebdc586afec9d13972de9e0d26e6"
server_key_iv: "b5ec4f40d1cd639eb6570880ad664afb"
server_key_version: "encryptionkey@0"
I20260619 22:25:27.309656 23143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "cd0ef362870c4f59991bc9edc71e1b73"
format_stamp: "Formatted at 2026-06-19 22:25:27 on dist-test-slave-k84h"
server_key: "8ec8ebdc586afec9d13972de9e0d26e6"
server_key_iv: "b5ec4f40d1cd639eb6570880ad664afb"
server_key_version: "encryptionkey@0"
I20260619 22:25:27.313429 23143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260619 22:25:27.316015 23158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:27.317278 23143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:27.317451 23143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "cd0ef362870c4f59991bc9edc71e1b73"
format_stamp: "Formatted at 2026-06-19 22:25:27 on dist-test-slave-k84h"
server_key: "8ec8ebdc586afec9d13972de9e0d26e6"
server_key_iv: "b5ec4f40d1cd639eb6570880ad664afb"
server_key_version: "encryptionkey@0"
I20260619 22:25:27.317582 23143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:27.322325 23143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:27.325724 23143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:27.325954 23143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:27.332954 23143 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:36211
I20260619 22:25:27.333048 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:36211 every 8 connection(s)
I20260619 22:25:27.333976 23143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:27.335847 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23143
I20260619 22:25:27.336021 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:27.336321 1322 external_mini_cluster.cc:1428] Setting key a4e2c1f67240d4e3fb1358f4b4270ccc
I20260619 22:25:27.337191 23211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:27.342298 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:27.337830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35399 (local address 127.1.74.190:36211)
0619 22:25:27.338283 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:27.338302 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:27.338347 (+ 45us) server_negotiation.cc:407] Connection header received
0619 22:25:27.339258 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:27.339282 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:27.339661 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:27.340061 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:27.340862 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:27.340868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:27.340883 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:27.340898 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:27.340979 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:27.341009 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:27.341154 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:27.341221 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 22:25:27.341375 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":141,"threads_started":1}
I20260619 22:25:27.344358 23211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73: Bootstrap starting.
I20260619 22:25:27.345453 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:27.347272 23211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:27.348215 23211 log.cc:824] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:27.349730 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:27.347823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33415 (local address 127.1.74.190:36211)
0619 22:25:27.347999 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:27.348003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:27.348194 (+ 191us) server_negotiation.cc:407] Connection header received
0619 22:25:27.348351 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:27.348357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:27.348401 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:27.348472 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:27.349332 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:27.349335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:27.349337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:27.349347 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:27.349373 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:27.349383 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:27.349453 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:27.349549 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 22:25:27.349581 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:25:27.350682 23211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73: No bootstrap required, opened a new log
I20260619 22:25:27.354375 23211 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0ef362870c4f59991bc9edc71e1b73" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36211 } }
I20260619 22:25:27.354645 23211 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:27.354718 23211 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0ef362870c4f59991bc9edc71e1b73, State: Initialized, Role: FOLLOWER
I20260619 22:25:27.355235 23211 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0ef362870c4f59991bc9edc71e1b73" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36211 } }
I20260619 22:25:27.355365 23211 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:27.355463 23211 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:27.355585 23211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:27.356756 23211 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0ef362870c4f59991bc9edc71e1b73" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36211 } }
I20260619 22:25:27.357146 23211 leader_election.cc:302] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0ef362870c4f59991bc9edc71e1b73; no voters:
I20260619 22:25:27.357498 23211 leader_election.cc:288] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:27.357715 23221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:27.357995 23221 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [term 1 LEADER]: Becoming Leader. State: Replica: cd0ef362870c4f59991bc9edc71e1b73, State: Running, Role: LEADER
I20260619 22:25:27.358343 23221 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0ef362870c4f59991bc9edc71e1b73" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36211 } }
I20260619 22:25:27.358958 23211 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:27.360138 23222 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd0ef362870c4f59991bc9edc71e1b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0ef362870c4f59991bc9edc71e1b73" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36211 } } }
I20260619 22:25:27.360298 23222 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:27.360662 23223 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0ef362870c4f59991bc9edc71e1b73. Latest consensus state: current_term: 1 leader_uuid: "cd0ef362870c4f59991bc9edc71e1b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0ef362870c4f59991bc9edc71e1b73" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 36211 } } }
I20260619 22:25:27.360831 23223 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:27.360749 23230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:27.364310 23230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:27.370784 23230 catalog_manager.cc:1347] Generated new cluster ID: 675919fcfec942049c92d14a3a1ef01f
I20260619 22:25:27.370914 23230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:27.391181 23230 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:27.391582 23230 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:27.397457 23230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd0ef362870c4f59991bc9edc71e1b73: Generated new TSK 0
I20260619 22:25:27.398367 23230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:27.411823 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23143
2026-06-19T22:25:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:27 dist-test-slave-k84h krb5kdc[23242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:27 dist-test-slave-k84h krb5kdc[23242](info): set up 2 sockets
Jun 19 22:25:27 dist-test-slave-k84h krb5kdc[23242](info): commencing operation
krb5kdc: starting...
W20260619 22:25:28.459342 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:28 dist-test-slave-k84h krb5kdc[23242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:28.586498 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33131
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:28.701000 23255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:28.701301 23255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:28.701345 23255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:28.705749 23255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:28.705827 23255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:28.705847 23255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:28.705864 23255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:28.705910 23255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:28.711568 23255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:33131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:28.712739 23255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:28.713793 23255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:28.722645 23260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:28.722635 23261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:28.722635 23263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:28.723032 23255 server_base.cc:1034] running on GCE node
I20260619 22:25:28.723594 23255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:28.724736 23255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:28.725935 23255 hybrid_clock.cc:648] HybridClock initialized: now 1781907928725926 us; error 33 us; skew 500 ppm
Jun 19 22:25:28 dist-test-slave-k84h krb5kdc[23242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907928, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:28.728830 23255 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:28.729852 23255 webserver.cc:460] Webserver started at http://127.1.74.190:44293/ using document root <none> and password file <none>
I20260619 22:25:28.730479 23255 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:28.730577 23255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:28.730819 23255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:28.733042 23255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "481b73e6e3b74a1b94d35516f63e1042"
format_stamp: "Formatted at 2026-06-19 22:25:28 on dist-test-slave-k84h"
server_key: "5989ff4c1ccf33ff2681d2311c0e4839"
server_key_iv: "45da0624206559993f988f0e4885a45d"
server_key_version: "encryptionkey@0"
I20260619 22:25:28.733570 23255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "481b73e6e3b74a1b94d35516f63e1042"
format_stamp: "Formatted at 2026-06-19 22:25:28 on dist-test-slave-k84h"
server_key: "5989ff4c1ccf33ff2681d2311c0e4839"
server_key_iv: "45da0624206559993f988f0e4885a45d"
server_key_version: "encryptionkey@0"
I20260619 22:25:28.737354 23255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:28.739984 23270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:28.741247 23255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:28.741362 23255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "481b73e6e3b74a1b94d35516f63e1042"
format_stamp: "Formatted at 2026-06-19 22:25:28 on dist-test-slave-k84h"
server_key: "5989ff4c1ccf33ff2681d2311c0e4839"
server_key_iv: "45da0624206559993f988f0e4885a45d"
server_key_version: "encryptionkey@0"
I20260619 22:25:28.741467 23255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:28.766177 23255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:28.769073 23255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:28.769230 23255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:28.776865 23255 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:33131
I20260619 22:25:28.776886 23322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:33131 every 8 connection(s)
I20260619 22:25:28.777815 23255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:28.780548 23323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:28.784780 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23255
I20260619 22:25:28.784932 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:28.785235 1322 external_mini_cluster.cc:1428] Setting key 73a3d56636e519d50cabf81b36246213
I20260619 22:25:28.786618 23323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042: Bootstrap starting.
I20260619 22:25:28.788897 23323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:28.789635 23323 log.cc:824] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:28.791630 23323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042: No bootstrap required, opened a new log
Jun 19 22:25:28 dist-test-slave-k84h krb5kdc[23242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:28.795367 23323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481b73e6e3b74a1b94d35516f63e1042" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33131 } }
I20260619 22:25:28.795670 23323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:28.795746 23323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481b73e6e3b74a1b94d35516f63e1042, State: Initialized, Role: FOLLOWER
I20260619 22:25:28.796288 23323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481b73e6e3b74a1b94d35516f63e1042" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33131 } }
I20260619 22:25:28.796411 23323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:28.796502 23323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:28.796619 23323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:28.797585 23323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481b73e6e3b74a1b94d35516f63e1042" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33131 } }
I20260619 22:25:28.797958 23323 leader_election.cc:302] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 481b73e6e3b74a1b94d35516f63e1042; no voters:
I20260619 22:25:28.798310 23323 leader_election.cc:288] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:28.798372 23328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:28.798580 23328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [term 1 LEADER]: Becoming Leader. State: Replica: 481b73e6e3b74a1b94d35516f63e1042, State: Running, Role: LEADER
I20260619 22:25:28.798918 23328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481b73e6e3b74a1b94d35516f63e1042" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33131 } }
I20260619 22:25:28.799686 23323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:28.800259 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:28.786731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54309 (local address 127.1.74.190:33131)
0619 22:25:28.787192 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:28.787207 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:28.787242 (+ 35us) server_negotiation.cc:407] Connection header received
0619 22:25:28.787888 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:28.787903 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:28.788272 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:28.788628 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:28.789416 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:28.790459 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:28.791402 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:28.791928 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:28.792115 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:28.794634 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:28.794639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:28.794655 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:28.794683 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:28.796688 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:28.797143 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:28.797149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:28.797155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:28.797274 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:28.797569 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:28.797572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:28.797574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:28.798110 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:28.798324 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:28.799017 (+ 693us) server_negotiation.cc:299] Negotiation successful
0619 22:25:28.799336 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":138,"threads_started":1}
I20260619 22:25:28.800573 23330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "481b73e6e3b74a1b94d35516f63e1042" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481b73e6e3b74a1b94d35516f63e1042" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33131 } } }
I20260619 22:25:28.800714 23330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:28.801095 23337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:28.801980 23329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 481b73e6e3b74a1b94d35516f63e1042. Latest consensus state: current_term: 1 leader_uuid: "481b73e6e3b74a1b94d35516f63e1042" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "481b73e6e3b74a1b94d35516f63e1042" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 33131 } } }
I20260619 22:25:28.802083 23329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:28.804075 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:28.804543 23337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:28.811704 23337 catalog_manager.cc:1347] Generated new cluster ID: dba80aee211b44ee90ec5085a05b9905
I20260619 22:25:28.811808 23337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:28.820397 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:28.806399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50065 (local address 127.1.74.190:33131)
0619 22:25:28.806541 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:28.806546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:28.812111 (+ 5565us) server_negotiation.cc:407] Connection header received
0619 22:25:28.812292 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:28.812297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:28.812348 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:28.812423 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:28.813249 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:28.813788 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:28.815011 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:28.815297 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:28.815420 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:28.816916 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:28.816920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:28.816923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:28.816960 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:28.818460 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:28.819037 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:28.819043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:28.819046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:28.819157 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:28.819459 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:28.819462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:28.819464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:28.819666 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:28.819815 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:28.820062 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 22:25:28.820187 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:25:28.828290 23337 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:28.829635 23337 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:28.841198 23337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 481b73e6e3b74a1b94d35516f63e1042: Generated new TSK 0
I20260619 22:25:28.842142 23337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:28.851913 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23255
2026-06-19T22:25:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:28 dist-test-slave-k84h krb5kdc[23354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:28 dist-test-slave-k84h krb5kdc[23354](info): set up 2 sockets
Jun 19 22:25:28 dist-test-slave-k84h krb5kdc[23354](info): commencing operation
krb5kdc: starting...
W20260619 22:25:29.895522 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:29 dist-test-slave-k84h krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:30.028620 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44769
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:44769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:30.140442 23367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:30.140838 23367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:30.140939 23367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:30.145836 23367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:30.145946 23367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:30.145987 23367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:30.146025 23367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:30.146060 23367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:30.151855 23367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:44769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:44769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:30.153021 23367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:30.154137 23367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:30.162962 23375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:30.163091 23373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:30.163064 23372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:30.163151 23367 server_base.cc:1034] running on GCE node
I20260619 22:25:30.163928 23367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:30.165167 23367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:30.166364 23367 hybrid_clock.cc:648] HybridClock initialized: now 1781907930166342 us; error 50 us; skew 500 ppm
Jun 19 22:25:30 dist-test-slave-k84h krb5kdc[23354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907930, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:30.169492 23367 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:30.170612 23367 webserver.cc:460] Webserver started at http://127.1.74.190:43719/ using document root <none> and password file <none>
I20260619 22:25:30.171308 23367 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:30.171422 23367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:30.171679 23367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:30.174018 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "7295f53862394a229210d09cd42f7151"
format_stamp: "Formatted at 2026-06-19 22:25:30 on dist-test-slave-k84h"
server_key: "d45c163ed6b18a4cee5cf0c687ca0e4c"
server_key_iv: "c1069f6d9da2be1776b46878334c6b41"
server_key_version: "encryptionkey@0"
I20260619 22:25:30.174705 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "7295f53862394a229210d09cd42f7151"
format_stamp: "Formatted at 2026-06-19 22:25:30 on dist-test-slave-k84h"
server_key: "d45c163ed6b18a4cee5cf0c687ca0e4c"
server_key_iv: "c1069f6d9da2be1776b46878334c6b41"
server_key_version: "encryptionkey@0"
I20260619 22:25:30.178946 23367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:25:30.181664 23382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:30.183048 23367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 22:25:30.183195 23367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "7295f53862394a229210d09cd42f7151"
format_stamp: "Formatted at 2026-06-19 22:25:30 on dist-test-slave-k84h"
server_key: "d45c163ed6b18a4cee5cf0c687ca0e4c"
server_key_iv: "c1069f6d9da2be1776b46878334c6b41"
server_key_version: "encryptionkey@0"
I20260619 22:25:30.183322 23367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:30.215633 23367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:30.218685 23367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:30.218940 23367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:30.226032 23367 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:44769
I20260619 22:25:30.226058 23434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:44769 every 8 connection(s)
I20260619 22:25:30.227027 23367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:30.229615 23435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:30.234921 23435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151: Bootstrap starting.
I20260619 22:25:30.235780 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23367
I20260619 22:25:30.235915 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:30.236222 1322 external_mini_cluster.cc:1428] Setting key fe763c14fc9ba066c476daecade02466
I20260619 22:25:30.237797 23435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:30.238500 23435 log.cc:824] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:30.240402 23435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151: No bootstrap required, opened a new log
I20260619 22:25:30.243973 23435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7295f53862394a229210d09cd42f7151" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44769 } }
I20260619 22:25:30.244223 23435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:30.244288 23435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7295f53862394a229210d09cd42f7151, State: Initialized, Role: FOLLOWER
I20260619 22:25:30.244768 23435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7295f53862394a229210d09cd42f7151" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44769 } }
I20260619 22:25:30.244923 23435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:30.245015 23435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:30.245132 23435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:30.245307 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:30.237361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55241 (local address 127.1.74.190:44769)
0619 22:25:30.237800 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:30.237814 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:30.237877 (+ 63us) server_negotiation.cc:407] Connection header received
0619 22:25:30.238624 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:30.238651 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:30.239088 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:30.239486 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:30.239894 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:30.240912 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:30.241800 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:30.242287 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:30.242487 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:30.243411 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:30.243424 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:30.243438 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:30.243822 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:30.243878 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:30.243907 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:30.244022 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:30.244172 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 22:25:30.244417 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":127,"threads_started":1}
I20260619 22:25:30.246301 23435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7295f53862394a229210d09cd42f7151" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44769 } }
I20260619 22:25:30.246639 23435 leader_election.cc:302] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7295f53862394a229210d09cd42f7151; no voters:
I20260619 22:25:30.247021 23435 leader_election.cc:288] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:30.247138 23440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:30.247342 23440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [term 1 LEADER]: Becoming Leader. State: Replica: 7295f53862394a229210d09cd42f7151, State: Running, Role: LEADER
I20260619 22:25:30.247689 23440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7295f53862394a229210d09cd42f7151" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44769 } }
I20260619 22:25:30.248160 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:30.248258 23435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:30.249293 23442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7295f53862394a229210d09cd42f7151" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7295f53862394a229210d09cd42f7151" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44769 } } }
I20260619 22:25:30.249421 23442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:30.249809 23441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7295f53862394a229210d09cd42f7151. Latest consensus state: current_term: 1 leader_uuid: "7295f53862394a229210d09cd42f7151" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7295f53862394a229210d09cd42f7151" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 44769 } } }
I20260619 22:25:30.249914 23441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151 [sys.catalog]: This master's current role is: LEADER
W20260619 22:25:30.253198 23459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:30.253289 23459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:30.253371 23453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:30.254226 23453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:30.255405 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:30.251148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34441 (local address 127.1.74.190:44769)
0619 22:25:30.251240 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:30.251245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:30.251426 (+ 181us) server_negotiation.cc:407] Connection header received
0619 22:25:30.251572 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:30.251575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:30.251622 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:30.251705 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:30.252017 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:30.252737 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:30.253787 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:30.254010 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:30.254088 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:30.254989 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:30.254993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:30.254996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:30.255008 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:30.255037 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:30.255051 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:30.255130 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:30.255231 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 22:25:30.255279 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 22:25:30.259985 23453 catalog_manager.cc:1347] Generated new cluster ID: 6ed8789250ba427d93f51ac4d4ee8a39
I20260619 22:25:30.260056 23453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:30.270484 23453 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:30.271770 23453 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:30.277841 23453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7295f53862394a229210d09cd42f7151: Generated new TSK 0
I20260619 22:25:30.278592 23453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:30.281836 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23367
2026-06-19T22:25:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:30 dist-test-slave-k84h krb5kdc[23466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:30 dist-test-slave-k84h krb5kdc[23466](info): set up 2 sockets
Jun 19 22:25:30 dist-test-slave-k84h krb5kdc[23466](info): commencing operation
krb5kdc: starting...
W20260619 22:25:31.327533 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:31 dist-test-slave-k84h krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:31.458256 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42753
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:42753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:31.568063 23479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:31.568390 23479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:31.568475 23479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:31.572546 23479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:31.572656 23479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:31.572696 23479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:31.572754 23479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:31.572793 23479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:31.578183 23479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:42753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:42753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:31.579298 23479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:31.580423 23479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:31.588871 23487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:31.588831 23485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:31.588994 23484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:31.589094 23479 server_base.cc:1034] running on GCE node
I20260619 22:25:31.589666 23479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:31.590827 23479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:31.592046 23479 hybrid_clock.cc:648] HybridClock initialized: now 1781907931592027 us; error 30 us; skew 500 ppm
Jun 19 22:25:31 dist-test-slave-k84h krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907931, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:31.594740 23479 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:31.595635 23479 webserver.cc:460] Webserver started at http://127.1.74.190:34741/ using document root <none> and password file <none>
I20260619 22:25:31.596276 23479 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:31.596375 23479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:31.596608 23479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:31.598661 23479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "7c993f7202ca4124ad21a277920b1554"
format_stamp: "Formatted at 2026-06-19 22:25:31 on dist-test-slave-k84h"
server_key: "78394599d5a541a7ed8df957eb013c62"
server_key_iv: "c14975910109c5738c829c99e7983399"
server_key_version: "encryptionkey@0"
I20260619 22:25:31.599278 23479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "7c993f7202ca4124ad21a277920b1554"
format_stamp: "Formatted at 2026-06-19 22:25:31 on dist-test-slave-k84h"
server_key: "78394599d5a541a7ed8df957eb013c62"
server_key_iv: "c14975910109c5738c829c99e7983399"
server_key_version: "encryptionkey@0"
I20260619 22:25:31.602984 23479 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 22:25:31.605289 23494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:31.606462 23479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 22:25:31.606617 23479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "7c993f7202ca4124ad21a277920b1554"
format_stamp: "Formatted at 2026-06-19 22:25:31 on dist-test-slave-k84h"
server_key: "78394599d5a541a7ed8df957eb013c62"
server_key_iv: "c14975910109c5738c829c99e7983399"
server_key_version: "encryptionkey@0"
I20260619 22:25:31.606735 23479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:31.611388 23479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:31.614279 23479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:31.614482 23479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:31.621730 23479 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:42753
I20260619 22:25:31.621754 23546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:42753 every 8 connection(s)
I20260619 22:25:31.622690 23479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:31.624541 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23479
I20260619 22:25:31.624687 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:31.624989 1322 external_mini_cluster.cc:1428] Setting key 52136fb3ff8f6b8dc7a7d37dc12b1648
I20260619 22:25:31.625830 23547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:31.630183 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:31.626374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37221 (local address 127.1.74.190:42753)
0619 22:25:31.626876 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:31.626897 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:31.626944 (+ 47us) server_negotiation.cc:407] Connection header received
0619 22:25:31.627587 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:31.627602 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:31.627875 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:31.628212 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:31.629025 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:31.629029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:31.629041 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:31.629052 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:31.629104 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:31.629122 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:31.629275 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:31.629345 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 22:25:31.629508 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":173,"threads_started":1}
I20260619 22:25:31.631961 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554: Bootstrap starting.
I20260619 22:25:31.633003 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:31.634411 23547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:31.635278 23547 log.cc:824] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:31.636906 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:31.635186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44619 (local address 127.1.74.190:42753)
0619 22:25:31.635327 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:31.635331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:31.635460 (+ 129us) server_negotiation.cc:407] Connection header received
0619 22:25:31.635628 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:31.635632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:31.635673 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:31.635758 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:31.636507 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:31.636511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:31.636513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 22:25:31.636525 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:31.636551 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 22:25:31.636570 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:31.636658 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:31.636742 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 22:25:31.636782 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:25:31.637305 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554: No bootstrap required, opened a new log
I20260619 22:25:31.640928 23547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c993f7202ca4124ad21a277920b1554" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42753 } }
I20260619 22:25:31.641209 23547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 22:25:31.641283 23547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c993f7202ca4124ad21a277920b1554, State: Initialized, Role: FOLLOWER
I20260619 22:25:31.641705 23547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c993f7202ca4124ad21a277920b1554" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42753 } }
I20260619 22:25:31.641842 23547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 22:25:31.641927 23547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 22:25:31.642050 23547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:31.643159 23547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c993f7202ca4124ad21a277920b1554" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42753 } }
I20260619 22:25:31.643448 23547 leader_election.cc:302] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c993f7202ca4124ad21a277920b1554; no voters:
I20260619 22:25:31.643702 23547 leader_election.cc:288] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 22:25:31.643831 23557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:31.644079 23557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [term 1 LEADER]: Becoming Leader. State: Replica: 7c993f7202ca4124ad21a277920b1554, State: Running, Role: LEADER
I20260619 22:25:31.644385 23557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c993f7202ca4124ad21a277920b1554" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42753 } }
I20260619 22:25:31.644937 23547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:31.646020 23559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c993f7202ca4124ad21a277920b1554. Latest consensus state: current_term: 1 leader_uuid: "7c993f7202ca4124ad21a277920b1554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c993f7202ca4124ad21a277920b1554" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42753 } } }
I20260619 22:25:31.646149 23559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:31.646445 23558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c993f7202ca4124ad21a277920b1554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c993f7202ca4124ad21a277920b1554" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 42753 } } }
I20260619 22:25:31.646553 23566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:31.649827 23566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:31.646579 23558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:31.655627 23566 catalog_manager.cc:1347] Generated new cluster ID: 38714b51090948fcac3f01decce75b89
I20260619 22:25:31.655700 23566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:31.675194 23566 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:31.675585 23566 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:31.687372 23566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c993f7202ca4124ad21a277920b1554: Generated new TSK 0
I20260619 22:25:31.688138 23566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 22:25:31.694344 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23479
2026-06-19T22:25:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1411 ms)
[----------] 18 tests from AuthTokenIssuingTest (25893 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:31 dist-test-slave-k84h krb5kdc[23578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:31 dist-test-slave-k84h krb5kdc[23578](info): set up 2 sockets
Jun 19 22:25:31 dist-test-slave-k84h krb5kdc[23578](info): commencing operation
krb5kdc: starting...
W20260619 22:25:32.739295 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:32 dist-test-slave-k84h krb5kdc[23578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:32.866072 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39559
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39559,127.1.74.189:42811,127.1.74.188:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:32.976975 23591 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 22:25:32.977301 23591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:32.977326 23591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:32.977367 23591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:32.981365 23591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:32.981426 23591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:32.981446 23591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:32.981465 23591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:32.981480 23591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:32.987349 23591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39559,127.1.74.189:42811,127.1.74.188:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:39559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:32.988463 23591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:32.989490 23591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:25:32.998087 23591 server_base.cc:1034] running on GCE node
W20260619 22:25:32.998044 23596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:32.998059 23597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:32.998050 23599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:32.998778 23591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:32.999989 23591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:33.001159 23591 hybrid_clock.cc:648] HybridClock initialized: now 1781907933001133 us; error 41 us; skew 500 ppm
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:33.008910 23591 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:33.009874 23591 webserver.cc:460] Webserver started at http://127.1.74.190:43331/ using document root <none> and password file <none>
I20260619 22:25:33.010468 23591 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:33.010532 23591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:33.010758 23591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:33.012843 23591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "ca1d4e845944482f86335348a1178cad"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "300f6df420431272163bc23a552dccdc"
server_key_iv: "bf6f99454e8ffa17d7e0e89657a80549"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.013422 23591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "ca1d4e845944482f86335348a1178cad"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "300f6df420431272163bc23a552dccdc"
server_key_iv: "bf6f99454e8ffa17d7e0e89657a80549"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.017269 23591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:33.019793 23606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:33.021040 23591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 22:25:33.021183 23591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "ca1d4e845944482f86335348a1178cad"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "300f6df420431272163bc23a552dccdc"
server_key_iv: "bf6f99454e8ffa17d7e0e89657a80549"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.021298 23591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:33.050494 23591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:33.053989 23591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:33.054205 23591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:33.061388 23591 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:39559
I20260619 22:25:33.061699 23658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:39559 every 8 connection(s)
I20260619 22:25:33.062412 23591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:33.063212 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23591
I20260619 22:25:33.063369 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:33.063692 1322 external_mini_cluster.cc:1428] Setting key 1a2547de0a6938583c11e8107f07e6f6
I20260619 22:25:33.066197 23659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:33.070436 23659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.189@KRBTEST.COM" created.
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:33.088279 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.073791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.1.74.190:39559)
0619 22:25:33.074099 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.074109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.074146 (+ 37us) server_negotiation.cc:407] Connection header received
0619 22:25:33.075028 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.075036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.075318 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.075546 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.076380 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.077239 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.078795 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.079413 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.079549 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.082537 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.082552 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.082564 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.082586 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.084574 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.085193 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.085199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.085201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.085319 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.085686 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.085689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.085691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.086128 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.086281 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.086763 (+ 482us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.087192 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":167,"server-negotiator.queue_time_us":195,"thread_start_us":123,"threads_started":1}
I20260619 22:25:33.088279 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.072993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39559 (local address 127.0.0.1:52612)
0619 22:25:33.073630 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.073676 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.074675 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.075554 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.075558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.075938 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.076199 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.076222 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.077364 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.077369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.078657 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.078672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.079591 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.079598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.079743 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.080410 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.080438 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.082350 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.084742 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.084754 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.084765 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.085070 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.085435 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.085438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.085440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.085578 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.086263 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.086270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.086452 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.086697 (+ 245us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.087139 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":4,"thread_start_us":143,"threads_started":1}
I20260619 22:25:33.090715 23659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.091552 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.091261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49088)
0619 22:25:33.091401 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.091486 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 22:25:33.091867 23610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
I20260619 22:25:33.092842 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.092637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49100)
0619 22:25:33.092740 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.092787 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 22:25:33.093056 23659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:42811: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab.
Entry for principal kudu/127.1.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab.
I20260619 22:25:33.094048 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:42811
--webserver_interface=127.1.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab
--principal=kudu/127.1.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39559,127.1.74.189:42811,127.1.74.188:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:25:33.142341 23659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } attempt: 1
I20260619 22:25:33.143318 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.143023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49114)
0619 22:25:33.143168 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.143241 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 22:25:33.144085 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.143861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49124)
0619 22:25:33.143963 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.144016 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 22:25:33.144263 23659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:42811: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
W20260619 22:25:33.205338 23666 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 22:25:33.205657 23666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:33.205680 23666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:33.205789 23666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:33.209964 23666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:33.210036 23666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:33.210062 23666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:33.210080 23666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:33.210152 23666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:33.212443 23659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } attempt: 2
I20260619 22:25:33.213336 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.213005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49132)
0619 22:25:33.213177 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.213242 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 22:25:33.214184 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.213900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49142)
0619 22:25:33.214065 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.214126 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 22:25:33.214408 23659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:42811: Network error: Client connection negotiation failed: client connection to 127.1.74.189:42811: connect: Connection refused (error 111)
I20260619 22:25:33.216400 23666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39559,127.1.74.189:42811,127.1.74.188:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab
--principal=kudu/127.1.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:33.217525 23666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:33.218556 23666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:33.227438 23674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:33.227485 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:33.227499 23666 server_base.cc:1034] running on GCE node
W20260619 22:25:33.227591 23672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:33.228238 23666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:33.229491 23666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:33.230831 23666 hybrid_clock.cc:648] HybridClock initialized: now 1781907933230805 us; error 35 us; skew 500 ppm
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:33.233860 23666 init.cc:377] Logged in from keytab as kudu/127.1.74.189@KRBTEST.COM (short username kudu)
I20260619 22:25:33.234786 23666 webserver.cc:460] Webserver started at http://127.1.74.189:40135/ using document root <none> and password file <none>
I20260619 22:25:33.235414 23666 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:33.235477 23666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:33.235647 23666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:33.237704 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data/instance:
uuid: "7d8592ec5ae44acca3491f5e7615c76f"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "296ff12a3d3690cd907e2e93d7dd2cac"
server_key_iv: "a2fafde5d911fdc53eaec902ea676041"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.238221 23666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal/instance:
uuid: "7d8592ec5ae44acca3491f5e7615c76f"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "296ff12a3d3690cd907e2e93d7dd2cac"
server_key_iv: "a2fafde5d911fdc53eaec902ea676041"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.242004 23666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:25:33.244592 23681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:33.245905 23666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 22:25:33.246024 23666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal
uuid: "7d8592ec5ae44acca3491f5e7615c76f"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "296ff12a3d3690cd907e2e93d7dd2cac"
server_key_iv: "a2fafde5d911fdc53eaec902ea676041"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.246119 23666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:33.285845 23666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:33.289198 23666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:33.289363 23666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:33.296396 23666 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.189:42811
I20260619 22:25:33.296684 23733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.189:42811 every 8 connection(s)
I20260619 22:25:33.297441 23666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
I20260619 22:25:33.300256 23734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:33.301214 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23666
I20260619 22:25:33.301321 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-1/wal/instance
I20260619 22:25:33.301671 1322 external_mini_cluster.cc:1428] Setting key 0345db00171cbae7ba5404b9fdf70686
I20260619 22:25:33.304657 23734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.305619 23659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } attempt: 3
WARNING: no policy specified for kudu/127.1.74.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.188@KRBTEST.COM" created.
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:33.320106 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.306160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49158)
0619 22:25:33.306315 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.306332 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.306425 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.308994 (+ 2569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.308997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.309208 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.309363 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.309373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.311606 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.311613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.312626 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.312641 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.313570 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.313573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.313696 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.314297 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.314312 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.315920 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.318018 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.318024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.318028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.318326 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.318736 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.318741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.318746 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.318875 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.319540 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.319543 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.319650 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.319773 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.319921 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 22:25:33.321625 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.306923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.1.74.189:42811)
0619 22:25:33.307414 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.307434 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.307479 (+ 45us) server_negotiation.cc:407] Connection header received
0619 22:25:33.308505 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.308532 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.308782 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.309535 (+ 753us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.309870 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.311450 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.312742 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.313434 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.313621 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.316089 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.316096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.316110 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.316136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.317896 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.318469 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.318477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.318484 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.318625 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.318976 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.318979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.318981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.319415 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.319574 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.319931 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.320501 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":322,"spinlock_wait_cycles":36224,"thread_start_us":143,"threads_started":1}
I20260619 22:25:33.322206 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.306943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.1.74.190:39559)
0619 22:25:33.307099 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.307106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.307641 (+ 535us) server_negotiation.cc:407] Connection header received
0619 22:25:33.308716 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.308719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.308773 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.308869 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.310938 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.311766 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.313270 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.314205 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.314320 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.317807 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.317811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.317818 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.317852 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.319694 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.320679 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.320682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.320684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.320769 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.321123 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.321126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.321128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.321335 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.321431 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.321879 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.322025 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 22:25:33.322759 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.306923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39559 (local address 127.0.0.1:52626)
0619 22:25:33.307513 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.307560 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.308512 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.308920 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.308928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.309356 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.310732 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.310747 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.311890 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.311896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.313123 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.313132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.314998 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.315002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.315133 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.315725 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.315746 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.317637 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.319836 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.319849 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.319856 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.320552 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.320871 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.320875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.320879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.321009 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.321457 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.321465 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.321607 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.322365 (+ 758us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.322584 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":796,"spinlock_wait_cycles":53632,"thread_start_us":154,"threads_started":1}
I20260619 22:25:33.324005 23659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.323979 23734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.324841 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.324555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56350)
0619 22:25:33.324710 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.324771 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 22:25:33.325663 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.325397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56360)
0619 22:25:33.325572 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.325614 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260619 22:25:33.325784 23659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:42721: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:33.332850 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.324787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49168)
0619 22:25:33.324876 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.324913 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.324999 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.325245 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.325251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.325465 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.325593 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.325599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.326415 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.326421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.327076 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.327085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.327546 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.327549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.327677 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.328180 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.328199 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.329524 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.331266 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.331272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.331275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.331492 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.331787 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.331792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.331794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.331875 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.332270 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.332273 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.332365 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.332479 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.332625 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":20}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab.
Entry for principal kudu/127.1.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab.
I20260619 22:25:33.332857 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.324938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.1.74.189:42811)
0619 22:25:33.325028 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.325034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.325049 (+ 15us) server_negotiation.cc:407] Connection header received
0619 22:25:33.325084 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.325087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.325130 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.325213 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.325704 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.326309 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.327196 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.327440 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.327549 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.329652 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.329655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.329658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.329687 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.331155 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.331589 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.331594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.331596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.331698 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.331968 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.331971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.331972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.332152 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.332281 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.332531 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.332675 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":25}
I20260619 22:25:33.333801 23734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.334154 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:42721
--webserver_interface=127.1.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab
--principal=kudu/127.1.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:46565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:39559,127.1.74.189:42811,127.1.74.188:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:25:33.334657 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.334287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56372)
0619 22:25:33.334474 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.334569 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260619 22:25:33.334939 23684 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
I20260619 22:25:33.335891 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.335646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56378)
0619 22:25:33.335780 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.335830 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 22:25:33.336140 23734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:42721: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
I20260619 22:25:33.357040 23659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } attempt: 1
I20260619 22:25:33.357969 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.357668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56388)
0619 22:25:33.357822 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.357889 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 22:25:33.358695 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.358483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56390)
0619 22:25:33.358583 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.358638 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 22:25:33.358819 23659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:42721: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
I20260619 22:25:33.385442 23734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } attempt: 1
I20260619 22:25:33.386405 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.386027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56396)
0619 22:25:33.386231 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.386309 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 22:25:33.387198 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.386974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56406)
0619 22:25:33.387086 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.387138 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 22:25:33.387347 23734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:42721: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
I20260619 22:25:33.433997 23659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } attempt: 2
I20260619 22:25:33.434957 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.434591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56422)
0619 22:25:33.434747 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.434816 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 22:25:33.435671 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.435456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56424)
0619 22:25:33.435561 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.435618 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 22:25:33.435817 23659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:42721: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
W20260619 22:25:33.447212 23741 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 22:25:33.447594 23741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:33.447661 23741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:33.447759 23741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:33.452275 23741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:33.452364 23741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:33.452404 23741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:33.452461 23741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:33.452499 23741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:33.455567 23734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } attempt: 2
I20260619 22:25:33.456462 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.456101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56436)
0619 22:25:33.456292 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.456367 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 22:25:33.457214 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.456990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56452)
0619 22:25:33.457098 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.457151 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 22:25:33.457345 23734 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:42721: Network error: Client connection negotiation failed: client connection to 127.1.74.188:42721: connect: Connection refused (error 111)
I20260619 22:25:33.458932 23741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:46565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:39559,127.1.74.189:42811,127.1.74.188:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab
--principal=kudu/127.1.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:42721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:33.460101 23741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:33.461125 23741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:33.470105 23749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:33.470139 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:33.470148 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:33.470286 23741 server_base.cc:1034] running on GCE node
I20260619 22:25:33.471279 23741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:33.472488 23741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:33.473733 23741 hybrid_clock.cc:648] HybridClock initialized: now 1781907933473720 us; error 31 us; skew 500 ppm
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:33.476859 23741 init.cc:377] Logged in from keytab as kudu/127.1.74.188@KRBTEST.COM (short username kudu)
I20260619 22:25:33.477841 23741 webserver.cc:460] Webserver started at http://127.1.74.188:35531/ using document root <none> and password file <none>
I20260619 22:25:33.478427 23741 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:33.478490 23741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:33.478664 23741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:33.480999 23741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data/instance:
uuid: "90eafb8d40e1466a9746bd94bcb0ba97"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "30de97611ccb8690ed845b3fe0763922"
server_key_iv: "5b2e465f6063096cd5fe2d8ceeb0ff66"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.481590 23741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal/instance:
uuid: "90eafb8d40e1466a9746bd94bcb0ba97"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "30de97611ccb8690ed845b3fe0763922"
server_key_iv: "5b2e465f6063096cd5fe2d8ceeb0ff66"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.485666 23741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 22:25:33.488399 23756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:33.489797 23741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:25:33.489956 23741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal
uuid: "90eafb8d40e1466a9746bd94bcb0ba97"
format_stamp: "Formatted at 2026-06-19 22:25:33 on dist-test-slave-k84h"
server_key: "30de97611ccb8690ed845b3fe0763922"
server_key_iv: "5b2e465f6063096cd5fe2d8ceeb0ff66"
server_key_version: "encryptionkey@0"
I20260619 22:25:33.490099 23741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:33.507597 23741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:33.510883 23741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:33.511117 23741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:33.518505 23741 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.188:42721
I20260619 22:25:33.518875 23808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.188:42721 every 8 connection(s)
I20260619 22:25:33.519625 23741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
I20260619 22:25:33.521299 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23741
I20260619 22:25:33.521430 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781907643212257-1322-0/minicluster-data/master-2/wal/instance
I20260619 22:25:33.521761 1322 external_mini_cluster.cc:1428] Setting key 1af4bd4b36e1acbac7ae7115ca5c1308
I20260619 22:25:33.522935 23809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:33.527526 23809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:33.534948 23659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } attempt: 3
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:33.537513 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.523208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.1.74.188:42721)
0619 22:25:33.523609 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.523625 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.523659 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:25:33.524289 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.524304 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.524632 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.524957 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.525883 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.527041 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.528217 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.528695 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.528828 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.531935 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.531940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.531950 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.531975 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.533850 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.534287 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.534293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.534300 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.534423 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.534714 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.534721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.534725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.535295 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.535485 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.536048 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.536385 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":131,"threads_started":1}
I20260619 22:25:33.540017 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.529384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.1.74.190:39559)
0619 22:25:33.529549 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.529554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.529952 (+ 398us) server_negotiation.cc:407] Connection header received
0619 22:25:33.530109 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.530112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.530155 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.530216 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.531098 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.531787 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.532832 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.533257 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.533378 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.535861 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.535866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.535870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.535896 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.537912 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.538437 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.538441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.538442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.538526 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.538879 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.538885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.538889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.539150 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.539259 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.539687 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.539798 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":52,"spinlock_wait_cycles":896}
I20260619 22:25:33.540544 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.529330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:39559 (local address 127.0.0.1:52642)
0619 22:25:33.529796 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.529853 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.529988 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.530308 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.530316 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.530770 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.530971 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.530977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.531909 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.531913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.532705 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.532715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.533401 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.533404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.533491 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.534029 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.534040 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.535698 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.538031 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.538040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.538045 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.538350 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.538649 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.538652 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.538653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.538745 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.539269 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.539277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.539420 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.540152 (+ 732us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.540342 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":158,"threads_started":1}
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:33.541654 23809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.545864 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.535403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56474)
0619 22:25:33.535696 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.535711 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.535788 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.536276 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.536279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.536450 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.536561 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.536567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.537457 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.537462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.538257 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.538271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.538941 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.538945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.539075 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.539567 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.539581 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.541118 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.544223 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.544227 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.544230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.544457 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.544775 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.544779 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.544781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.544861 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.545320 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.545326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.545432 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.545558 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.545690 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":23}
I20260619 22:25:33.545876 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.535684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56474 (local address 127.1.74.188:42721)
0619 22:25:33.536034 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.536043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.536056 (+ 13us) server_negotiation.cc:407] Connection header received
0619 22:25:33.536090 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.536092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.536142 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.536230 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.536675 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.537336 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.538382 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.538742 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.538941 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.541224 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.541227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.541230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.541251 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.544125 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.544567 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.544570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.544571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.544658 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.544999 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.545002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.545004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.545223 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.545331 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.545613 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.545721 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":111,"threads_started":1}
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:33.548473 23734 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } attempt: 3
I20260619 22:25:33.549615 23659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad: Bootstrap starting.
I20260619 22:25:33.551584 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.542399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.1.74.189:42811)
0619 22:25:33.542570 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.542576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.542590 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:25:33.542630 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.542633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.542679 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.542765 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.543195 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.543669 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.544655 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.545023 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.545150 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.547799 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.547807 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.547811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.547847 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.549629 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.550128 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.550132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.550134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.550216 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.550554 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.550557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.550595 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.550833 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.550939 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.551296 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.551407 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 22:25:33.552070 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.542148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:42811 (local address 127.0.0.1:49184)
0619 22:25:33.542289 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.542307 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.542381 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.542780 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.542782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.542987 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.543075 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.543081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.543774 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.543780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.544545 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.544554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.545147 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.545150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.545252 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.546044 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.546058 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.547645 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.549738 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.549743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.549748 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.550006 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.550341 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.550347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.550352 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.550442 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.551014 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.551017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.551109 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.551777 (+ 668us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.551905 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 22:25:33.552410 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.540159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.1.74.190:39559)
0619 22:25:33.540590 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.540595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.540607 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:25:33.540919 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.540922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.540970 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.541055 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.541789 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.542471 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.544018 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.544362 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.544460 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.547882 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.547886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.547888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.547912 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.549877 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.550371 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.550374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.550375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.550462 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.550825 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.550828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.550830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.551084 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.551241 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.551936 (+ 695us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.552039 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357}
I20260619 22:25:33.552557 23659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:33.552760 23809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:33.553651 23659 log.cc:824] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad: Log is configured to *not* fsync() on all Append() calls
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:33.556819 23659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad: No bootstrap required, opened a new log
I20260619 22:25:33.559020 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.549326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56476)
0619 22:25:33.549552 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.549568 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.549639 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.550290 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.550293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.550449 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.550561 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.550567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.551654 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.551658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.552399 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.552413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.552988 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.552994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.553079 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.553581 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.553594 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.555114 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.557135 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.557140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.557144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.557392 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.557703 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.557706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.557707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.557797 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.558507 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.558511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.558603 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.558684 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.558815 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907933, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:33.559957 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.549155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.1.74.188:42721)
0619 22:25:33.550063 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.550067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.550079 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:25:33.550111 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.550113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.550153 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.550668 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.550753 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.551527 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.552570 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.552884 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.552982 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.555253 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.555257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.555259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.555283 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.557011 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.557516 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.557519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.557521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.557600 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.557907 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.557910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.558171 (+ 261us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.558392 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.559489 (+ 1097us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.559639 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.559769 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":3456}
Jun 19 22:25:33 dist-test-slave-k84h krb5kdc[23578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:33.563205 23734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: Bootstrap starting.
I20260619 22:25:33.563546 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.553430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:42721 (local address 127.0.0.1:56492)
0619 22:25:33.554217 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:33.554231 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:33.554308 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:33.554601 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:33.554604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:33.554783 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:33.555315 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.555322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.556264 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.556268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.557079 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:33.557089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.557698 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.557701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:33.557771 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.558207 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:33.558221 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:33.559638 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:33.561771 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.561774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.561776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.562089 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.562408 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:33.562411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:33.562412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:33.562503 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:33.563029 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:33.563034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:33.563125 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 22:25:33.563230 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 22:25:33.563367 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"spinlock_wait_cycles":4480}
I20260619 22:25:33.563984 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.553549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.1.74.188:42721)
0619 22:25:33.553656 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.553660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.554386 (+ 726us) server_negotiation.cc:407] Connection header received
0619 22:25:33.554411 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.554414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.554449 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.554509 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.555423 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.556160 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.557182 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.557526 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.557610 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.559756 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.559760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.559762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.559786 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.561648 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.562210 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.562213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.562215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.562300 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.562610 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.562615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.562620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.562835 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.562994 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.563741 (+ 747us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.563844 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:25:33.565538 23659 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.565789 23659 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1d4e845944482f86335348a1178cad, State: Initialized, Role: FOLLOWER
I20260619 22:25:33.566111 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.555578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.1.74.189:42811)
0619 22:25:33.555716 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.555720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.555913 (+ 193us) server_negotiation.cc:407] Connection header received
0619 22:25:33.556087 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.556090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.556131 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.556177 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.557022 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.557734 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.558902 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.559245 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.559325 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.562480 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.562484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.562486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.562509 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.564214 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.564712 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.564715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.564717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.564797 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.565094 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.565097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.565099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.565307 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.565469 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.565867 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.565978 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 22:25:33.566141 23734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:33.566134 23659 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.567191 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: Bootstrap starting.
I20260619 22:25:33.567371 23734 log.cc:824] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:33.567656 23659 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:33.569411 23734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: No bootstrap required, opened a new log
I20260619 22:25:33.569716 23809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:33.570533 23809 log.cc:824] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:33.571563 23816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.571681 23816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: This master's current role is: FOLLOWER
W20260619 22:25:33.572686 23828 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:33.572769 23828 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:33.573043 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:33.574470 23633 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ca1d4e845944482f86335348a1178cad"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52648
I20260619 22:25:33.574687 23633 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 22:25:33.574796 23633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:33.576141 23633 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.572862 23591 master_runner.cc:186] Error getting master registration for 127.1.74.189:42811: OK, instance_id { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" instance_seqno: 1781907933294345 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 22:25:33.576823 23591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 22:25:33.576946 23734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.577170 23734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d8592ec5ae44acca3491f5e7615c76f, State: Initialized, Role: FOLLOWER
I20260619 22:25:33.577194 23633 leader_election.cc:288] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [CANDIDATE]: Term 1 election: Requested vote from peers 7d8592ec5ae44acca3491f5e7615c76f (127.1.74.189:42811), 90eafb8d40e1466a9746bd94bcb0ba97 (127.1.74.188:42721)
I20260619 22:25:33.577502 23734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.578070 23831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.578270 23831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:33.578910 23708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca1d4e845944482f86335348a1178cad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d8592ec5ae44acca3491f5e7615c76f"
I20260619 22:25:33.579126 23734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:33.579142 23708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:33.582590 23809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: No bootstrap required, opened a new log
I20260619 22:25:33.587484 23708 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1d4e845944482f86335348a1178cad in term 1.
I20260619 22:25:33.587779 23809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.588068 23809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90eafb8d40e1466a9746bd94bcb0ba97, State: Initialized, Role: FOLLOWER
I20260619 22:25:33.588030 23610 leader_election.cc:302] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d8592ec5ae44acca3491f5e7615c76f, ca1d4e845944482f86335348a1178cad; no voters:
I20260619 22:25:33.588388 23816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:33.588431 23809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.588653 23816 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [term 1 LEADER]: Becoming Leader. State: Replica: ca1d4e845944482f86335348a1178cad, State: Running, Role: LEADER
I20260619 22:25:33.589061 23816 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } }
I20260619 22:25:33.589960 23809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:33.590124 23783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca1d4e845944482f86335348a1178cad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "90eafb8d40e1466a9746bd94bcb0ba97"
I20260619 22:25:33.590363 23783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:33.591892 23816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca1d4e845944482f86335348a1178cad. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.591971 23783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca1d4e845944482f86335348a1178cad in term 1.
I20260619 22:25:33.592027 23816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:33.594331 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.580587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.1.74.189:42811)
0619 22:25:33.580703 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.580707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.581335 (+ 628us) server_negotiation.cc:407] Connection header received
0619 22:25:33.581551 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.581554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.581602 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.581687 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.582567 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.583335 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.584488 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.584847 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.584956 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.586760 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.586764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.586767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.586794 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.589109 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.589907 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.589911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.589913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.590013 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.592977 (+ 2964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.592984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.592988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.593526 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.593647 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.594053 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.594181 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 22:25:33.603088 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.581056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.1.74.188:42721)
0619 22:25:33.581169 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.581174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.593553 (+ 12379us) server_negotiation.cc:407] Connection header received
0619 22:25:33.593727 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.593730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.593777 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.593827 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.595275 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.596029 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.597132 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.597473 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.597545 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.599367 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.599371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.599374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.599401 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.601221 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.601760 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.601764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.601766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.601848 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.602124 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.602127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.602129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.602331 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.602449 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.602771 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.602937 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:25:33.603159 23858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:33.604184 23858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:33.606137 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:33.593174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52660 (local address 127.1.74.190:39559)
0619 22:25:33.593279 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.593283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.595025 (+ 1742us) server_negotiation.cc:407] Connection header received
0619 22:25:33.595199 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.595202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.595250 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.595301 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.596699 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.597393 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.598562 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:33.598945 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:33.599031 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:33.602358 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:33.602362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:33.602365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:33.602395 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:33.604308 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.604832 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.604835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.604837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.604926 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:33.605236 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:33.605239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:33.605241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:33.605462 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:33.605593 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:33.605868 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 22:25:33.605990 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 22:25:33.607035 23845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.607187 23845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 22:25:33.607663 23863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:33.607743 23863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260619 22:25:33.610536 23850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:33.610656 23850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:33.613104 23783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [term 1 FOLLOWER]: Refusing update from remote peer ca1d4e845944482f86335348a1178cad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:33.613126 23708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [term 1 FOLLOWER]: Refusing update from remote peer ca1d4e845944482f86335348a1178cad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:33.613834 23843 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:33.614369 23843 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [LEADER]: Connected to new peer: Peer: permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:33.618155 23831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca1d4e845944482f86335348a1178cad. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.618312 23831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:33.619400 23845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca1d4e845944482f86335348a1178cad. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.619575 23845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:33.624583 23866 mvcc.cc:205] Tried to move back new op lower bound from 7298694896070197248 to 7298694895984697344. Current Snapshot: MvccSnapshot[applied={T|T < 7298694896070197248}]
I20260619 22:25:33.625002 23816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.625147 23816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:33.627997 23831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.628149 23831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:33.629407 23869 mvcc.cc:205] Tried to move back new op lower bound from 7298694896070197248 to 7298694895984697344. Current Snapshot: MvccSnapshot[applied={T|T < 7298694896070197248}]
I20260619 22:25:33.631490 23858 catalog_manager.cc:1347] Generated new cluster ID: 9c7d4543e1f349dba9a11d0a0a13bc44
I20260619 22:25:33.631567 23858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:33.635316 23843 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.635489 23843 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:33.636742 23845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca1d4e845944482f86335348a1178cad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1d4e845944482f86335348a1178cad" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 39559 } } peers { permanent_uuid: "7d8592ec5ae44acca3491f5e7615c76f" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 42811 } } peers { permanent_uuid: "90eafb8d40e1466a9746bd94bcb0ba97" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 42721 } } }
I20260619 22:25:33.636881 23845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:33.640038 23870 mvcc.cc:205] Tried to move back new op lower bound from 7298694896070197248 to 7298694895984697344. Current Snapshot: MvccSnapshot[applied={T|T < 7298694896070197248}]
I20260619 22:25:33.656040 23858 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:33.657449 23858 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:33.693226 23858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca1d4e845944482f86335348a1178cad: Generated new TSK 0
I20260619 22:25:33.694276 23858 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 22:25:33.733726 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.730453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.1.74.189:42811)
0619 22:25:33.730577 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.730582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.732638 (+ 2056us) server_negotiation.cc:407] Connection header received
0619 22:25:33.732832 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.732836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.732886 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.732950 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.733589 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49208: BlockingRecv error: recv got EOF from 127.0.0.1:49208 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 22:25:33.735121 23812 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.731196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.1.74.188:42721)
0619 22:25:33.731307 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.731312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.734204 (+ 2892us) server_negotiation.cc:407] Connection header received
0619 22:25:33.734365 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.734368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.734417 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.734466 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.734968 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56504: BlockingRecv error: recv got EOF from 127.0.0.1:56504 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 22:25:33.736034 23663 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.732036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.1.74.190:39559)
0619 22:25:33.732149 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.732153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.735082 (+ 2929us) server_negotiation.cc:407] Connection header received
0619 22:25:33.735277 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.735280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.735343 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.735410 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.735922 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52674: BlockingRecv error: recv got EOF from 127.0.0.1:52674 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 22:25:33.744364 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.739300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.1.74.189:42811)
0619 22:25:33.739417 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.739421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.743183 (+ 3762us) server_negotiation.cc:407] Connection header received
0619 22:25:33.743436 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.743440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.743516 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.743575 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.744255 (+ 680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49218: BlockingRecv error: recv got EOF from 127.0.0.1:49218 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 22:25:33.751247 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.749551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.1.74.189:42811)
0619 22:25:33.749662 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.749666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.750223 (+ 557us) server_negotiation.cc:407] Connection header received
0619 22:25:33.750425 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.750429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.750480 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.750529 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.751137 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49230: BlockingRecv error: recv got EOF from 127.0.0.1:49230 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 22:25:33.757105 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.755515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.1.74.189:42811)
0619 22:25:33.755619 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.755624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.756132 (+ 508us) server_negotiation.cc:407] Connection header received
0619 22:25:33.756316 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.756319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.756372 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.756437 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.757002 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49234: BlockingRecv error: recv got EOF from 127.0.0.1:49234 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 22:25:33.765345 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.763713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.1.74.189:42811)
0619 22:25:33.763844 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.763848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.764367 (+ 519us) server_negotiation.cc:407] Connection header received
0619 22:25:33.764582 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.764586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.764640 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.764697 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.765239 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49238: BlockingRecv error: recv got EOF from 127.0.0.1:49238 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 22:25:33.777678 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.775897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.1.74.189:42811)
0619 22:25:33.776023 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.776028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.776648 (+ 620us) server_negotiation.cc:407] Connection header received
0619 22:25:33.776852 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.776855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.776912 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.776969 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.777559 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49250: BlockingRecv error: recv got EOF from 127.0.0.1:49250 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 22:25:33.797750 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.796141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.1.74.189:42811)
0619 22:25:33.796264 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.796269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.796826 (+ 557us) server_negotiation.cc:407] Connection header received
0619 22:25:33.797017 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.797021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.797072 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.797122 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.797634 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49260: BlockingRecv error: recv got EOF from 127.0.0.1:49260 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 22:25:33.833443 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.832159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.1.74.189:42811)
0619 22:25:33.832302 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.832313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.832479 (+ 166us) server_negotiation.cc:407] Connection header received
0619 22:25:33.832647 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.832649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.832698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.832770 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.833290 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49274: BlockingRecv error: recv got EOF from 127.0.0.1:49274 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 22:25:33.900796 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:33.899574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.1.74.189:42811)
0619 22:25:33.899750 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:33.899755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:33.899883 (+ 128us) server_negotiation.cc:407] Connection header received
0619 22:25:33.900076 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:33.900078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:33.900128 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:33.900220 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:33.900682 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49290: BlockingRecv error: recv got EOF from 127.0.0.1:49290 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 22:25:34.032250 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:34.031038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.1.74.189:42811)
0619 22:25:34.031205 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.031209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.031366 (+ 157us) server_negotiation.cc:407] Connection header received
0619 22:25:34.031501 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.031503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.031553 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.031640 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:34.032101 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49306: BlockingRecv error: recv got EOF from 127.0.0.1:49306 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 22:25:34.292029 23737 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:34.290713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.1.74.189:42811)
0619 22:25:34.290885 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.290890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.291079 (+ 189us) server_negotiation.cc:407] Connection header received
0619 22:25:34.291286 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.291289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.291336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.291429 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:34.291893 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49314: BlockingRecv error: recv got EOF from 127.0.0.1:49314 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 22:25:34.609086 23863 catalog_manager.cc:1259] Loaded cluster ID: 9c7d4543e1f349dba9a11d0a0a13bc44
I20260619 22:25:34.609177 23863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: loading cluster ID for follower catalog manager: success
I20260619 22:25:34.610767 23863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: acquiring CA information for follower catalog manager: success
I20260619 22:25:34.611971 23850 catalog_manager.cc:1259] Loaded cluster ID: 9c7d4543e1f349dba9a11d0a0a13bc44
I20260619 22:25:34.612056 23850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: loading cluster ID for follower catalog manager: success
I20260619 22:25:34.612210 23863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90eafb8d40e1466a9746bd94bcb0ba97: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:25:34.613648 23850 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: acquiring CA information for follower catalog manager: success
I20260619 22:25:34.614717 23850 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d8592ec5ae44acca3491f5e7615c76f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:25:34.810894 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:34.806194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.1.74.189:42811)
0619 22:25:34.806484 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.806488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.806568 (+ 80us) server_negotiation.cc:407] Connection header received
0619 22:25:34.806724 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.806728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.806916 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.807045 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:34.807279 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.807936 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.808864 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.809108 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.809271 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:34.809714 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:34.810395 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:34.810542 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:34.810620 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 22:25:34.810685 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260619 22:25:34.817993 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:34.814074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56518 (local address 127.1.74.188:42721)
0619 22:25:34.814277 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.814281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.814295 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:25:34.814457 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.814459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.814572 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.814675 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:34.815017 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.815558 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.816483 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.816713 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.816801 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:34.817123 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:34.817565 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:34.817660 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:34.817734 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 22:25:34.817785 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260619 22:25:34.826097 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:34.821854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.1.74.188:42721)
0619 22:25:34.822018 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.822023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.822035 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:25:34.822085 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.822087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.822153 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.822224 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:34.822552 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.823422 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.824741 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.825066 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.825163 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:34.825507 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:34.825600 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:34.825678 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:34.825837 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 22:25:34.825898 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 22:25:34.826164 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:34.821645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.1.74.190:39559)
0619 22:25:34.821871 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.821877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.821893 (+ 16us) server_negotiation.cc:407] Connection header received
0619 22:25:34.822009 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.822012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.822200 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.822358 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:34.822610 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.823442 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.824458 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.824865 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.824965 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:34.825335 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:34.825689 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:34.825778 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:34.825933 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 22:25:34.826000 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
I20260619 22:25:34.826507 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:34.821390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.1.74.189:42811)
0619 22:25:34.821540 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:34.821546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:34.822134 (+ 588us) server_negotiation.cc:407] Connection header received
0619 22:25:34.822234 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:34.822237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:34.822291 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:34.822397 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:34.822719 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.823326 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.824775 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:34.825174 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:34.825272 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:34.826010 (+ 738us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:34.826092 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:34.826139 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:34.826319 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 22:25:34.826378 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 22:25:34.829692 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23591
I20260619 22:25:34.836037 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23666
I20260619 22:25:34.842308 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23741
2026-06-19T22:25:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3150 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 22:25:34 dist-test-slave-k84h krb5kdc[23958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 22:25:34 dist-test-slave-k84h krb5kdc[23958](info): set up 2 sockets
Jun 19 22:25:34 dist-test-slave-k84h krb5kdc[23958](info): commencing operation
krb5kdc: starting...
W20260619 22:25:35.891537 1322 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/test-user.keytab.
Jun 19 22:25:35 dist-test-slave-k84h krb5kdc[23958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T22:25:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T22:25:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.74.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
Entry for principal kudu/127.1.74.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab.
I20260619 22:25:36.028399 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37513
--webserver_interface=127.1.74.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37513,127.1.74.189:40223,127.1.74.188:39715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:36.158782 23971 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 22:25:36.159132 23971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:36.159157 23971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:36.159271 23971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:36.163870 23971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:36.163964 23971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:36.163985 23971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:36.164003 23971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:36.164085 23971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:36.169750 23971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37513,127.1.74.189:40223,127.1.74.188:39715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.190.keytab
--principal=kudu/127.1.74.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.190:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.74.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:36.170964 23971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:36.172024 23971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:36.180955 23976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:36.180979 23971 server_base.cc:1034] running on GCE node
W20260619 22:25:36.180922 23977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:36.180929 23979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:36.181829 23971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:36.183000 23971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:36.184166 23971 hybrid_clock.cc:648] HybridClock initialized: now 1781907936184156 us; error 26 us; skew 500 ppm
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:36.187006 23971 init.cc:377] Logged in from keytab as kudu/127.1.74.190@KRBTEST.COM (short username kudu)
I20260619 22:25:36.187846 23971 webserver.cc:460] Webserver started at http://127.1.74.190:33671/ using document root <none> and password file <none>
I20260619 22:25:36.188377 23971 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:36.188436 23971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:36.188594 23971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:36.190573 23971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data/instance:
uuid: "89613c750dff4f048dd344c8948aa448"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "8dd0d03e1830450c4bbc49d65395b7d1"
server_key_iv: "fe4e24f77fba01701cc38e90f767f7d1"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.191164 23971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal/instance:
uuid: "89613c750dff4f048dd344c8948aa448"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "8dd0d03e1830450c4bbc49d65395b7d1"
server_key_iv: "fe4e24f77fba01701cc38e90f767f7d1"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.194944 23971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 22:25:36.197218 23986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:36.198437 23971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 22:25:36.198590 23971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal
uuid: "89613c750dff4f048dd344c8948aa448"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "8dd0d03e1830450c4bbc49d65395b7d1"
server_key_iv: "fe4e24f77fba01701cc38e90f767f7d1"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.198701 23971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:36.228011 23971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:36.231345 23971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:36.231576 23971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:36.238691 23971 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.190:37513
I20260619 22:25:36.239032 24038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.190:37513 every 8 connection(s)
I20260619 22:25:36.239809 23971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/data/info.pb
I20260619 22:25:36.242669 24039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:36.246410 24039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:36.247195 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 23971
I20260619 22:25:36.247361 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-0/wal/instance
I20260619 22:25:36.247666 1322 external_mini_cluster.cc:1428] Setting key a7fafa14321a6f26619663fc79bf9dfb
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.74.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.189@KRBTEST.COM" created.
I20260619 22:25:36.263535 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.248902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37513 (local address 127.0.0.1:43900)
0619 22:25:36.249431 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.249489 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.250378 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.251241 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.251245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.251595 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.251844 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.251856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.253818 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.253821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.254558 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.254567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.255691 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.255694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.255823 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.256447 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.256463 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.258305 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.260373 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.260416 (+ 43us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.260424 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.260660 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.260995 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.260997 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.260998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.261164 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.261815 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.261825 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.262054 (+ 229us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.262249 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.262604 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":43,"thread_start_us":169,"threads_started":1}
I20260619 22:25:36.263542 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.248903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.1.74.190:37513)
0619 22:25:36.249545 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.249560 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.249618 (+ 58us) server_negotiation.cc:407] Connection header received
0619 22:25:36.250724 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.250730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.251045 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.251213 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.252011 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.253685 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.254701 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.255521 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.255723 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.258482 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.258489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.258500 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.258523 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.260225 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.260756 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.260761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.260763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.260865 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.261243 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.261249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.261254 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.261679 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.261777 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.262315 (+ 538us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.262525 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":499,"spinlock_wait_cycles":8960,"thread_start_us":188,"threads_started":1}
I20260619 22:25:36.266276 24039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:36.267174 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.266827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41854)
0619 22:25:36.267045 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.267112 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260619 22:25:36.267467 23987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
I20260619 22:25:36.268306 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.268111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41860)
0619 22:25:36.268214 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.268257 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 22:25:36.268599 24039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:40223: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab.
Entry for principal kudu/127.1.74.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab.
I20260619 22:25:36.279110 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:40223
--webserver_interface=127.1.74.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab
--principal=kudu/127.1.74.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37513,127.1.74.189:40223,127.1.74.188:39715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 22:25:36.317934 24039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } attempt: 1
I20260619 22:25:36.318979 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.318583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41864)
0619 22:25:36.318760 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.318825 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 22:25:36.319723 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.319512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41866)
0619 22:25:36.319619 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.319671 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 22:25:36.319852 24039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:40223: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
I20260619 22:25:36.388039 24039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } attempt: 2
I20260619 22:25:36.389055 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.388704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41878)
0619 22:25:36.388867 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.388939 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 22:25:36.389842 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.389614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41890)
0619 22:25:36.389726 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.389782 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 22:25:36.389958 24039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.189:40223: Network error: Client connection negotiation failed: client connection to 127.1.74.189:40223: connect: Connection refused (error 111)
W20260619 22:25:36.396998 24046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 22:25:36.397363 24046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:36.397440 24046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:36.397521 24046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 22:25:36.401840 24046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:36.401933 24046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:36.401970 24046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:36.402025 24046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:36.402063 24046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:36.407858 24046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37513,127.1.74.189:40223,127.1.74.188:39715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.189.keytab
--principal=kudu/127.1.74.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.189:40223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.74.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:36.408995 24046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:36.410022 24046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 22:25:36.418792 24054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:36.418823 24052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:36.418763 24051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:36.419118 24046 server_base.cc:1034] running on GCE node
I20260619 22:25:36.419777 24046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:36.420974 24046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:36.422181 24046 hybrid_clock.cc:648] HybridClock initialized: now 1781907936422146 us; error 50 us; skew 500 ppm
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:36.425264 24046 init.cc:377] Logged in from keytab as kudu/127.1.74.189@KRBTEST.COM (short username kudu)
I20260619 22:25:36.426216 24046 webserver.cc:460] Webserver started at http://127.1.74.189:44125/ using document root <none> and password file <none>
I20260619 22:25:36.426801 24046 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:36.426918 24046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:36.427150 24046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:36.429200 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data/instance:
uuid: "e157d4d2310e4e3f8364aa27506460c8"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "b2bde469850edb28c3c25959ad459e48"
server_key_iv: "a5c62739838850e9e354efdc0ecdfd11"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.429761 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal/instance:
uuid: "e157d4d2310e4e3f8364aa27506460c8"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "b2bde469850edb28c3c25959ad459e48"
server_key_iv: "a5c62739838850e9e354efdc0ecdfd11"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.433655 24046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 22:25:36.436389 24061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:36.437879 24046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 22:25:36.438057 24046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal
uuid: "e157d4d2310e4e3f8364aa27506460c8"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "b2bde469850edb28c3c25959ad459e48"
server_key_iv: "a5c62739838850e9e354efdc0ecdfd11"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.438198 24046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:36.468744 24046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:36.472251 24046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:36.472474 24046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:36.480221 24046 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.189:40223
I20260619 22:25:36.480535 24113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.189:40223 every 8 connection(s)
I20260619 22:25:36.481144 24039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } attempt: 3
I20260619 22:25:36.481396 24046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/data/info.pb
I20260619 22:25:36.484669 24114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:36.487221 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 24046
I20260619 22:25:36.487478 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-1/wal/instance
I20260619 22:25:36.487771 1322 external_mini_cluster.cc:1428] Setting key 9897ce43af24f102e9e87373876fb462
I20260619 22:25:36.489609 24114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:36.496558 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.481749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41902)
0619 22:25:36.481930 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.481951 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.482081 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.484546 (+ 2465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.484550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.484778 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.484939 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.484949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.486409 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.486413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.487297 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.487311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.488362 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.488369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.488534 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.489194 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.489217 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.490686 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.494452 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.494457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.494461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.494762 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.495214 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.495217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.495219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.495316 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.495989 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.495993 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.496105 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.496219 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.496375 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
WARNING: no policy specified for kudu/127.1.74.188@KRBTEST.COM; defaulting to no policy
I20260619 22:25:36.497634 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.482617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.1.74.189:40223)
0619 22:25:36.483074 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.483095 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.483141 (+ 46us) server_negotiation.cc:407] Connection header received
0619 22:25:36.483854 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.483871 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.484229 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.484534 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.485196 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.486244 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.487460 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.488195 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.488378 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.490937 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.490944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.490960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.490994 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.494319 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.494915 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.494921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.494927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.495130 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.495422 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.495425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.495427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.495863 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.496010 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.496370 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.496823 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":174,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.74.188@KRBTEST.COM" created.
I20260619 22:25:36.500010 24039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:36.500934 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.500547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54050)
0619 22:25:36.500771 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.500834 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260619 22:25:36.501816 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.501457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54060)
0619 22:25:36.501671 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.501729 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260619 22:25:36.501935 24039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:39715: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
I20260619 22:25:36.505323 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.491907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.1.74.190:37513)
0619 22:25:36.492042 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.492047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.492636 (+ 589us) server_negotiation.cc:407] Connection header received
0619 22:25:36.492797 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.492804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.492867 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.492988 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.494319 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.495041 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.496439 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.497473 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.497583 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.501142 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.501148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.501150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.501175 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.503174 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.503839 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.503842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.503843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.503963 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.504271 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.504273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.504274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.504461 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.504548 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.505021 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.505176 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 22:25:36.505483 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.491901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37513 (local address 127.0.0.1:43912)
0619 22:25:36.492462 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.492530 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.492685 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.493028 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.493038 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.493944 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.494158 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.494168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.495180 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.495186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.496283 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.496296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.497694 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.497699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.497852 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.498487 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.498501 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.500949 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.503308 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.503318 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.503323 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.503731 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.504063 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.504070 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.504075 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.504180 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.504572 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.504582 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.504746 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.504923 (+ 177us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.505169 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":159,"threads_started":1}
I20260619 22:25:36.506647 24114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:36.516700 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.507202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41906)
0619 22:25:36.507348 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.507364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.507455 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.507693 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.507696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.507888 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.508018 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.508025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.508835 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.508841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.509773 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.509784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.510324 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.510327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.510431 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.511132 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.511155 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.513054 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.515081 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.515084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.515086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.515352 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.515634 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.515637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.515639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.515721 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.516127 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.516129 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.516212 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.516323 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.516501 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 22:25:36.516700 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.507292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.1.74.189:40223)
0619 22:25:36.507404 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.507415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.507427 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:25:36.507549 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.507551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.507586 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.507651 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.508176 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.508705 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.509922 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.510184 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.510280 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.513222 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.513225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.513227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.513248 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.514943 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.515447 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.515451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.515456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.515532 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.515823 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.515826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.515827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.516031 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.516141 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.516398 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.516559 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":40}
I20260619 22:25:36.517684 24114 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.74.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab.
Entry for principal kudu/127.1.74.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab.
I20260619 22:25:36.518710 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.518271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54074)
0619 22:25:36.518500 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.518616 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 22:25:36.519213 1322 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSWejqq/build/debug/bin/kudu
/tmp/dist-test-taskSWejqq/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:39715
--webserver_interface=127.1.74.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab
--principal=kudu/127.1.74.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.74.148:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.74.190:37513,127.1.74.189:40223,127.1.74.188:39715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 22:25:36.519410 24063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
I20260619 22:25:36.520485 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.520146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54086)
0619 22:25:36.520304 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.520389 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 22:25:36.520676 24114 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:39715: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
I20260619 22:25:36.533152 24039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } attempt: 1
I20260619 22:25:36.534080 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.533774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54100)
0619 22:25:36.533930 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.533997 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 22:25:36.534826 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.534609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54112)
0619 22:25:36.534712 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.534769 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 22:25:36.535079 24039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:39715: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
I20260619 22:25:36.570003 24114 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } attempt: 1
I20260619 22:25:36.570988 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.570621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54128)
0619 22:25:36.570768 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.570894 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 22:25:36.571879 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.571634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54134)
0619 22:25:36.571759 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.571812 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 22:25:36.572063 24114 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:39715: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
I20260619 22:25:36.610306 24039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } attempt: 2
I20260619 22:25:36.611232 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.610933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54136)
0619 22:25:36.611073 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.611143 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 22:25:36.612037 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.611806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54150)
0619 22:25:36.611920 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.611971 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 22:25:36.612211 24039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:39715: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
W20260619 22:25:36.638479 24121 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 22:25:36.638797 24121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 22:25:36.638821 24121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 22:25:36.638945 24121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 22:25:36.640235 24114 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } attempt: 2
I20260619 22:25:36.641114 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.640805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54156)
0619 22:25:36.640967 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.641034 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 22:25:36.641991 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.641728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54158)
0619 22:25:36.641822 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.641868 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 22:25:36.642189 24114 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.74.188:39715: Network error: Client connection negotiation failed: client connection to 127.1.74.188:39715: connect: Connection refused (error 111)
W20260619 22:25:36.644562 24121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 22:25:36.644621 24121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 22:25:36.644641 24121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 22:25:36.644690 24121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 22:25:36.644721 24121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 22:25:36.650398 24121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.74.148:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.74.190:37513,127.1.74.189:40223,127.1.74.188:39715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/krb5kdc/kudu_127.1.74.188.keytab
--principal=kudu/127.1.74.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.74.188:39715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.74.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 21:32:11 UTC on ccycloud.quasar-utnohq.root.comops.site
build id 4456676
I20260619 22:25:36.651638 24121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 22:25:36.652729 24121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 22:25:36.661763 24121 server_base.cc:1034] running on GCE node
W20260619 22:25:36.661751 24127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:36.661743 24126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 22:25:36.661937 24129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 22:25:36.662679 24121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 22:25:36.663911 24121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 22:25:36.665110 24121 hybrid_clock.cc:648] HybridClock initialized: now 1781907936665097 us; error 29 us; skew 500 ppm
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 22:25:36.668362 24121 init.cc:377] Logged in from keytab as kudu/127.1.74.188@KRBTEST.COM (short username kudu)
I20260619 22:25:36.669355 24121 webserver.cc:460] Webserver started at http://127.1.74.188:46635/ using document root <none> and password file <none>
I20260619 22:25:36.669952 24121 fs_manager.cc:359] Metadata directory not provided
I20260619 22:25:36.670027 24121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 22:25:36.670257 24121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 22:25:36.672592 24121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data/instance:
uuid: "eb37ef52f4834778a61f96d79f2d7180"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "8661059489b38e1de0814a5115ae7050"
server_key_iv: "11db629a045c366a37c2b0efa39e0830"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.673207 24121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal/instance:
uuid: "eb37ef52f4834778a61f96d79f2d7180"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "8661059489b38e1de0814a5115ae7050"
server_key_iv: "11db629a045c366a37c2b0efa39e0830"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.677475 24121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 22:25:36.680097 24136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 22:25:36.681394 24121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260619 22:25:36.681540 24121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data,/tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal
uuid: "eb37ef52f4834778a61f96d79f2d7180"
format_stamp: "Formatted at 2026-06-19 22:25:36 on dist-test-slave-k84h"
server_key: "8661059489b38e1de0814a5115ae7050"
server_key_iv: "11db629a045c366a37c2b0efa39e0830"
server_key_version: "encryptionkey@0"
I20260619 22:25:36.681675 24121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 22:25:36.694823 24121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 22:25:36.698113 24121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 22:25:36.698323 24121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 22:25:36.705797 24121 rpc_server.cc:307] RPC server started. Bound to: 127.1.74.188:39715
I20260619 22:25:36.706135 24188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.74.188:39715 every 8 connection(s)
I20260619 22:25:36.707001 24121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/data/info.pb
I20260619 22:25:36.709842 24189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 22:25:36.711386 24039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } attempt: 3
I20260619 22:25:36.714243 24189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:36.716583 1322 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSWejqq/build/debug/bin/kudu as pid 24121
I20260619 22:25:36.716717 1322 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSWejqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781907643212257-1322-0/minicluster-data/master-2/wal/instance
I20260619 22:25:36.716990 1322 external_mini_cluster.cc:1428] Setting key ac4b2fbea399a437caab607b3f845a7a
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.190@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:36.725536 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.711923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54164)
0619 22:25:36.712086 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.712104 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.712206 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.714554 (+ 2348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.714558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.714767 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.714925 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.714934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.716317 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.716322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.717155 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.717167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.718168 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.718171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.718252 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.718815 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.718870 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.720369 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.723085 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.723088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.723091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.723441 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.723970 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.723975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.723978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.724080 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.724885 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.724888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.725015 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.725212 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.725363 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 22:25:36.726970 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.712691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54164 (local address 127.1.74.188:39715)
0619 22:25:36.713162 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.713178 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.713212 (+ 34us) server_negotiation.cc:407] Connection header received
0619 22:25:36.713878 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.713895 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.714244 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.714528 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.715077 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.716144 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.717316 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.718035 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.718218 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.720505 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.720513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.720530 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.720571 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.722962 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.723589 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.723623 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.723635 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.723831 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.724206 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.724208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.724210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.724762 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.724985 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.725374 (+ 389us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.725856 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":151,"threads_started":1}
I20260619 22:25:36.727313 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.716016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.190:37513 (local address 127.0.0.1:43918)
0619 22:25:36.716441 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.716490 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.716653 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.716997 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.717002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.717392 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.717570 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.717579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.718571 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.718575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.719371 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.719381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.720038 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.720045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.720168 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.720813 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.720827 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.722799 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.725040 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.725049 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.725055 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.725380 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.725916 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.725919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.725920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.726043 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.726526 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.726535 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.726708 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.726914 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.727126 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260619 22:25:36.727547 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.716143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.1.74.190:37513)
0619 22:25:36.716317 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.716322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.716567 (+ 245us) server_negotiation.cc:407] Connection header received
0619 22:25:36.716769 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.716771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.716812 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.716853 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.717708 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.718411 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.719517 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.719861 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.719964 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.723028 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.723034 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.723041 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.723074 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.724868 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.725506 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.725509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.725511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.725606 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.726176 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.726180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.726181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.726400 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.726502 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.726998 (+ 496us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.727377 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 22:25:36.729388 24189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:36.729832 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.717647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.1.74.188:39715)
0619 22:25:36.718341 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.718348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.718360 (+ 12us) server_negotiation.cc:407] Connection header received
0619 22:25:36.718389 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.718391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.718431 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.718509 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.719555 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.720276 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.721852 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.722207 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.722374 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.725538 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.725544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.725550 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.725577 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.728027 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.728481 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.728487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.728489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.728570 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.728905 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.728910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.728912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.729098 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.729279 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.729518 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.729633 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":138,"threads_started":1}
I20260619 22:25:36.732512 24039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448: Bootstrap starting.
I20260619 22:25:36.733356 24114 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } attempt: 3
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:36.736012 24039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:36.737003 24039 log.cc:824] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:36.738659 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.730081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.189:40223 (local address 127.0.0.1:41918)
0619 22:25:36.730446 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.730459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.730541 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.730795 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.730798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.731018 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.731186 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.731194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.732007 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.732010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.732513 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.732519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.733028 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.733032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.733131 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.733652 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.733668 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.734984 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.736984 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.736987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.736989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.737234 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.737580 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.737582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.737584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.737683 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.738163 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.738166 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.738269 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.738374 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.738510 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293}
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.190@KRBTEST.COM
I20260619 22:25:36.739145 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.730181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.1.74.189:40223)
0619 22:25:36.730306 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.730311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.730529 (+ 218us) server_negotiation.cc:407] Connection header received
0619 22:25:36.730642 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.730645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.730694 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.730783 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.731333 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.731894 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.732689 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.732917 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.733030 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.735123 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.735126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.735129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.735154 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.736855 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.737355 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.737361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.737366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.737469 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.737779 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.737782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.737784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.738009 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.738084 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.738481 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.738616 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":47}
I20260619 22:25:36.739663 24039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448: No bootstrap required, opened a new log
I20260619 22:25:36.740165 24189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } has no permanent_uuid. Determining permanent_uuid...
I20260619 22:25:36.744109 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.731163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.1.74.190:37513)
0619 22:25:36.734065 (+ 2902us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.734069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.734079 (+ 10us) server_negotiation.cc:407] Connection header received
0619 22:25:36.734106 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.734108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.734141 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.734228 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.734978 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.735625 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.736580 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.736821 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.736928 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.739650 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.739653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.739656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.739686 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.741858 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.742436 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.742440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.742441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.742543 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.742895 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.742898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.742900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.743184 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.743385 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.743632 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.743751 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2839}
I20260619 22:25:36.746328 24039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.746575 24039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89613c750dff4f048dd344c8948aa448, State: Initialized, Role: FOLLOWER
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.188@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:36.747045 24039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907936, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.74.189@KRBTEST.COM for kudu/127.1.74.188@KRBTEST.COM
I20260619 22:25:36.748768 24039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:36.751567 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.741094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.1.74.188:39715)
0619 22:25:36.741232 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.741237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.741434 (+ 197us) server_negotiation.cc:407] Connection header received
0619 22:25:36.741564 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.741567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.741608 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.741693 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.742153 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.742992 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.744027 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.744614 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.744752 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.747606 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.747610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.747612 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.747632 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.749380 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.749975 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.749978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.749980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.750059 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.750341 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.750344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.750345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.750582 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.750687 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.751300 (+ 613us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.751409 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":56}
I20260619 22:25:36.751726 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.741189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54186)
0619 22:25:36.741357 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.741374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.741471 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.741731 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.741737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.741931 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.742045 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.742053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.743136 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.743139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.743915 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.743922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.744757 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.744760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.744865 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.745396 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.745418 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.747476 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.749497 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.749501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.749506 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.749882 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.750174 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.750177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.750178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.750253 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.750725 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.750731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.750817 (+ 86us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.750950 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.751100 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":3}
I20260619 22:25:36.752125 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.733745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.74.188:39715 (local address 127.0.0.1:54174)
0619 22:25:36.733842 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 22:25:36.733858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 22:25:36.733939 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 22:25:36.737763 (+ 3824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 22:25:36.737769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 22:25:36.737946 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 22:25:36.738032 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.738041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.743897 (+ 5856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.743901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.744606 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 22:25:36.744618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.745551 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.745558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 22:25:36.745640 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.746113 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 22:25:36.746126 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 22:25:36.747687 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 22:25:36.750282 (+ 2595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.750286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.750288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.750583 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.750970 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 22:25:36.750974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 22:25:36.750976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 22:25:36.751072 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 22:25:36.751569 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 22:25:36.751572 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 22:25:36.751680 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 22:25:36.751790 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 22:25:36.751947 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 22:25:36.753458 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.733786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.1.74.188:39715)
0619 22:25:36.737523 (+ 3737us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.737530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.737544 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:25:36.737579 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.737582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.737625 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.742930 (+ 5305us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.743014 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.743766 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.744755 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.745403 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.745850 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.747863 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.747868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.747871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.747899 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.750156 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.750717 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.750721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.750723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.750812 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.751191 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.751194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.751196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.751450 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.753082 (+ 1632us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.753171 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.753295 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":279,"server-negotiator.queue_time_us":3651,"spinlock_wait_cycles":3840}
W20260619 22:25:36.754395 24207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:36.754469 24207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:36.755317 24189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: Bootstrap starting.
I20260619 22:25:36.756562 24114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: Bootstrap starting.
I20260619 22:25:36.758208 24189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:36.759032 24114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: Neither blocks nor log segments found. Creating new log.
I20260619 22:25:36.754972 23971 master_runner.cc:186] Error getting master registration for 127.1.74.189:40223: OK, instance_id { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" instance_seqno: 1781907936477974 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 22:25:36.759208 23971 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 22:25:36.759212 24189 log.cc:824] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:36.759864 24114 log.cc:824] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: Log is configured to *not* fsync() on all Append() calls
I20260619 22:25:36.761693 24189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: No bootstrap required, opened a new log
Jun 19 22:25:36 dist-test-slave-k84h krb5kdc[23958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781907935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.74.189@KRBTEST.COM
I20260619 22:25:36.763051 24196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.763211 24196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.765254 24189 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.765497 24189 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb37ef52f4834778a61f96d79f2d7180, State: Initialized, Role: FOLLOWER
I20260619 22:25:36.765836 24189 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.766348 24211 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.766543 24211 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.767298 24189 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:36.767438 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.746117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41926 (local address 127.1.74.189:40223)
0619 22:25:36.746273 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.746287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.746483 (+ 196us) server_negotiation.cc:407] Connection header received
0619 22:25:36.746709 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.746715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.746773 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.746905 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.748642 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.749718 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.750930 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.751268 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.751344 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.763396 (+ 12052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.763400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.763403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.763449 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.765238 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.765778 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.765782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.765784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.765870 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.766179 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.766182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.766184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.766439 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.766616 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.767062 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.767205 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 22:25:36.771631 1322 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 22:25:36.772531 24114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: No bootstrap required, opened a new log
I20260619 22:25:36.772845 24012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "89613c750dff4f048dd344c8948aa448"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43934
I20260619 22:25:36.773099 24012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 22:25:36.773228 24012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 0 FOLLOWER]: Advancing to term 1
W20260619 22:25:36.773589 24222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 22:25:36.773677 24222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:36.774729 24012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.775851 24114 raft_consensus.cc:348] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.776089 24114 raft_consensus.cc:729] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e157d4d2310e4e3f8364aa27506460c8, State: Initialized, Role: FOLLOWER
I20260619 22:25:36.776288 24163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89613c750dff4f048dd344c8948aa448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eb37ef52f4834778a61f96d79f2d7180"
I20260619 22:25:36.776394 24012 leader_election.cc:288] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [CANDIDATE]: Term 1 election: Requested vote from peers e157d4d2310e4e3f8364aa27506460c8 (127.1.74.189:40223), eb37ef52f4834778a61f96d79f2d7180 (127.1.74.188:39715)
I20260619 22:25:36.776417 24114 consensus_queue.cc:260] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.776501 24163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:36.777822 24163 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89613c750dff4f048dd344c8948aa448 in term 1.
I20260619 22:25:36.778374 23988 leader_election.cc:302] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89613c750dff4f048dd344c8948aa448, eb37ef52f4834778a61f96d79f2d7180; no voters:
I20260619 22:25:36.778842 24196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 22:25:36.779140 24196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [term 1 LEADER]: Becoming Leader. State: Replica: 89613c750dff4f048dd344c8948aa448, State: Running, Role: LEADER
I20260619 22:25:36.779440 24196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } }
I20260619 22:25:36.779598 24224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.780143 24224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.781011 24196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89613c750dff4f048dd344c8948aa448. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.781117 24196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:36.781495 24232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 22:25:36.781811 24114 sys_catalog.cc:564] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 22:25:36.781860 24088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89613c750dff4f048dd344c8948aa448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e157d4d2310e4e3f8364aa27506460c8"
I20260619 22:25:36.782102 24088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 22:25:36.782732 24232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 22:25:36.783578 24088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89613c750dff4f048dd344c8948aa448 in term 1.
I20260619 22:25:36.784526 24121 master_runner.cc:186] Error getting master registration for 127.1.74.189:40223: OK, instance_id { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" instance_seqno: 1781907936477974 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 22:25:36.788168 24121 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 22:25:36.799042 24244 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 22:25:36.799132 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.779184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54196 (local address 127.1.74.188:39715)
0619 22:25:36.779302 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.779307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.788960 (+ 9653us) server_negotiation.cc:407] Connection header received
0619 22:25:36.789175 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.789178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.789233 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.789296 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.790461 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.791246 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.792466 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.792806 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.792891 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.795034 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.795040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.795046 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.795079 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.796976 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.797512 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.797516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.797518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.797619 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.797966 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.797969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.797971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.798204 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.798319 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.798779 (+ 460us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.798943 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 22:25:36.799144 24244 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 22:25:36.800951 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.780981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.1.74.190:37513)
0619 22:25:36.781100 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.781105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.791029 (+ 9924us) server_negotiation.cc:407] Connection header received
0619 22:25:36.791129 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.791132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.791182 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.791265 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.792182 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.792948 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.794504 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.794877 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.794949 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.797056 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.797060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.797063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.797097 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.798761 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.799632 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.799635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.799637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.799724 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.800019 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.800022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.800023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.800221 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.800340 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.800690 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.800806 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 22:25:36.802819 24163 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [term 1 FOLLOWER]: Refusing update from remote peer 89613c750dff4f048dd344c8948aa448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:36.803212 24088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [term 1 FOLLOWER]: Refusing update from remote peer 89613c750dff4f048dd344c8948aa448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 22:25:36.803850 24229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:36.804292 24196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 22:25:36.808349 24211 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89613c750dff4f048dd344c8948aa448. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.808496 24211 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.808635 24224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89613c750dff4f048dd344c8948aa448. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.808761 24224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.811688 24229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.811828 24229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:36.812861 24224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.812976 24224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.814457 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:36.788501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.1.74.189:40223)
0619 22:25:36.788605 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.788610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.799538 (+ 10928us) server_negotiation.cc:407] Connection header received
0619 22:25:36.800076 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.800080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.800137 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.800194 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.801241 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.801930 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.805564 (+ 3634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:36.805926 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:36.806017 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:36.809839 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 22:25:36.809843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 22:25:36.809845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 22:25:36.809873 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 22:25:36.812078 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.812819 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.812824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.812826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.812923 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 22:25:36.813362 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 22:25:36.813365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 22:25:36.813367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 22:25:36.813622 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 22:25:36.813755 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:36.814149 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 22:25:36.814274 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 22:25:36.817175 24249 mvcc.cc:205] Tried to move back new op lower bound from 7298694909135511552 to 7298694909051678720. Current Snapshot: MvccSnapshot[applied={T|T < 7298694909135511552}]
I20260619 22:25:36.818769 24232 catalog_manager.cc:1347] Generated new cluster ID: f63a8a14e80e40059bbe8522b1d69d44
I20260619 22:25:36.818832 24232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 22:25:36.840240 24196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.840473 24196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448 [sys.catalog]: This master's current role is: LEADER
I20260619 22:25:36.841845 24211 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89613c750dff4f048dd344c8948aa448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89613c750dff4f048dd344c8948aa448" member_type: VOTER last_known_addr { host: "127.1.74.190" port: 37513 } } peers { permanent_uuid: "e157d4d2310e4e3f8364aa27506460c8" member_type: VOTER last_known_addr { host: "127.1.74.189" port: 40223 } } peers { permanent_uuid: "eb37ef52f4834778a61f96d79f2d7180" member_type: VOTER last_known_addr { host: "127.1.74.188" port: 39715 } } }
I20260619 22:25:36.842010 24211 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 22:25:36.842535 24248 mvcc.cc:205] Tried to move back new op lower bound from 7298694909135511552 to 7298694909051678720. Current Snapshot: MvccSnapshot[applied={T|T < 7298694909135511552}]
I20260619 22:25:36.850257 24232 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 22:25:36.851691 24232 catalog_manager.cc:1504] Loading token signing keys...
I20260619 22:25:36.860558 24232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89613c750dff4f048dd344c8948aa448: Generated new TSK 0
I20260619 22:25:36.861433 24232 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 22:25:36.894498 24194 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.891276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.1.74.188:39715)
0619 22:25:36.891415 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.891420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.893281 (+ 1861us) server_negotiation.cc:407] Connection header received
0619 22:25:36.893501 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.893505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.893563 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.893621 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.894346 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54210: BlockingRecv error: recv got EOF from 127.0.0.1:54210 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 22:25:36.896390 24042 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.892060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.1.74.190:37513)
0619 22:25:36.892185 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.892189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.895085 (+ 2896us) server_negotiation.cc:407] Connection header received
0619 22:25:36.895270 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.895273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.895330 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.895389 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.896218 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43946: BlockingRecv error: recv got EOF from 127.0.0.1:43946 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 22:25:36.898082 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.892702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.1.74.189:40223)
0619 22:25:36.892818 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.892822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.897007 (+ 4185us) server_negotiation.cc:407] Connection header received
0619 22:25:36.897209 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.897212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.897274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.897333 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.897942 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41952: BlockingRecv error: recv got EOF from 127.0.0.1:41952 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 22:25:36.902628 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.901073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.1.74.189:40223)
0619 22:25:36.901193 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.901199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.901435 (+ 236us) server_negotiation.cc:407] Connection header received
0619 22:25:36.901714 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.901718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.901807 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.901881 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.902504 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41968: BlockingRecv error: recv got EOF from 127.0.0.1:41968 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 22:25:36.909096 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.907341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.1.74.189:40223)
0619 22:25:36.907469 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.907475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.908040 (+ 565us) server_negotiation.cc:407] Connection header received
0619 22:25:36.908254 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.908258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.908318 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.908377 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.908968 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41974: BlockingRecv error: recv got EOF from 127.0.0.1:41974 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 22:25:36.915756 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.913842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.1.74.189:40223)
0619 22:25:36.913981 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.913987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.914551 (+ 564us) server_negotiation.cc:407] Connection header received
0619 22:25:36.914781 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.914785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.914916 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.914981 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.915620 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41986: BlockingRecv error: recv got EOF from 127.0.0.1:41986 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 22:25:36.924813 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.922596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.1.74.189:40223)
0619 22:25:36.922768 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.922774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.923651 (+ 877us) server_negotiation.cc:407] Connection header received
0619 22:25:36.923897 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.923901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.923963 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.924024 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.924667 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42002: BlockingRecv error: recv got EOF from 127.0.0.1:42002 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260619 22:25:36.937820 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.935901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.1.74.189:40223)
0619 22:25:36.936038 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.936044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.936668 (+ 624us) server_negotiation.cc:407] Connection header received
0619 22:25:36.936914 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.936919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.936983 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.937045 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.937677 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42012: BlockingRecv error: recv got EOF from 127.0.0.1:42012 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 22:25:36.958724 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.956801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.1.74.189:40223)
0619 22:25:36.956935 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.956940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.957543 (+ 603us) server_negotiation.cc:407] Connection header received
0619 22:25:36.957815 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.957821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.957901 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.957967 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.958598 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42024: BlockingRecv error: recv got EOF from 127.0.0.1:42024 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 22:25:36.995080 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:36.993679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.1.74.189:40223)
0619 22:25:36.993809 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:36.993814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:36.994071 (+ 257us) server_negotiation.cc:407] Connection header received
0619 22:25:36.994145 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:36.994148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:36.994213 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:36.994326 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:36.994933 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42036: BlockingRecv error: recv got EOF from 127.0.0.1:42036 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 22:25:37.062711 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:37.061401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.1.74.189:40223)
0619 22:25:37.061593 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.061604 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.061762 (+ 158us) server_negotiation.cc:407] Connection header received
0619 22:25:37.061926 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.061930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.061996 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.062093 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:37.062561 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42038: BlockingRecv error: recv got EOF from 127.0.0.1:42038 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260619 22:25:37.194205 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:37.192920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.1.74.189:40223)
0619 22:25:37.193105 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.193111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.193281 (+ 170us) server_negotiation.cc:407] Connection header received
0619 22:25:37.193458 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.193462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.193526 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.193620 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:37.194079 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42052: BlockingRecv error: recv got EOF from 127.0.0.1:42052 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260619 22:25:37.454064 24116 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 22:25:37.452738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.1.74.189:40223)
0619 22:25:37.452914 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.452918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.452941 (+ 23us) server_negotiation.cc:407] Connection header received
0619 22:25:37.453146 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.453150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.453204 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.453282 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 22:25:37.453887 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42058: BlockingRecv error: recv got EOF from 127.0.0.1:42058 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 22:25:37.775038 24222 catalog_manager.cc:1259] Loaded cluster ID: f63a8a14e80e40059bbe8522b1d69d44
I20260619 22:25:37.775159 24222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: loading cluster ID for follower catalog manager: success
I20260619 22:25:37.776813 24222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: acquiring CA information for follower catalog manager: success
I20260619 22:25:37.777995 24222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb37ef52f4834778a61f96d79f2d7180: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:25:37.800484 24244 catalog_manager.cc:1259] Loaded cluster ID: f63a8a14e80e40059bbe8522b1d69d44
I20260619 22:25:37.800630 24244 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: loading cluster ID for follower catalog manager: success
I20260619 22:25:37.802354 24244 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: acquiring CA information for follower catalog manager: success
I20260619 22:25:37.803599 24244 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e157d4d2310e4e3f8364aa27506460c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 22:25:37.972711 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:37.968473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.1.74.189:40223)
0619 22:25:37.968751 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.968758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.968779 (+ 21us) server_negotiation.cc:407] Connection header received
0619 22:25:37.968982 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.968986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.969121 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.969251 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:37.969540 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.970195 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.971105 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.971339 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.971453 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:37.971791 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:37.972240 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:37.972385 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:37.972428 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 22:25:37.972496 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260619 22:25:37.980266 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:37.976250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.1.74.188:39715)
0619 22:25:37.976492 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.976496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.976510 (+ 14us) server_negotiation.cc:407] Connection header received
0619 22:25:37.976657 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.976660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.976787 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.976889 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:37.977193 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.977720 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.978654 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.978950 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.979034 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:37.979364 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:37.979812 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:37.979905 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:37.980003 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 22:25:37.980054 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260619 22:25:37.988355 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:37.984093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.1.74.188:39715)
0619 22:25:37.984240 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.984244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.984444 (+ 200us) server_negotiation.cc:407] Connection header received
0619 22:25:37.984587 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.984590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.984640 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.984715 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:37.985168 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.985838 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.987069 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.987431 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.987541 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:37.987793 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:37.987882 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:37.987949 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:37.988121 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 22:25:37.988190 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 22:25:37.988963 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:37.984353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.1.74.189:40223)
0619 22:25:37.984505 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.984509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.984851 (+ 342us) server_negotiation.cc:407] Connection header received
0619 22:25:37.985033 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.985037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.985105 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.985158 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:37.985558 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.986344 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.987645 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.988008 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.988114 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:37.988451 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:37.988538 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:37.988602 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:37.988748 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 22:25:37.988805 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 22:25:37.989264 24335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 22:25:37.983930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.1.74.190:37513)
0619 22:25:37.984200 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 22:25:37.984205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 22:25:37.984393 (+ 188us) server_negotiation.cc:407] Connection header received
0619 22:25:37.984604 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 22:25:37.984610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 22:25:37.984791 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 22:25:37.984922 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 22:25:37.985480 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.986310 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.987418 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 22:25:37.987800 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 22:25:37.987924 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 22:25:37.988418 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 22:25:37.988814 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 22:25:37.988904 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 22:25:37.989044 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 22:25:37.989118 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
I20260619 22:25:37.992479 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 23971
I20260619 22:25:37.999025 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 24046
I20260619 22:25:38.004786 1322 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSWejqq/build/debug/bin/kudu with pid 24121
2026-06-19T22:25:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3162 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6313 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (294785 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST